Deep Dive Articles

The Best Way to Learn a New Programming Language from Scratch - How I Approach and Learn Any Programming Language Efficiently

Preface: The following article is based on my experiences and opinions on learning programming languages. I have been learning about computers in-depth and find learning languages with specific purposes useful. For example, Python was for beginning, C is for memory management and Rust is for learning to write memory-safe code in my case. This time, I started learning Golang which is known for its simple syntax and powerful performance, and widely known for its awesome concurrency.

Why More is Not Always Good in Terms of Software - Words on Cross Platform Utilities, Bash-ism, and POSIX Compliance.

Preface: This article is about my views on software compliance and cross-platform support, and reflects my opinions and experience with the subject. Your experience and opinions may vary, which I respect. What I am Specifically Talking About? I am going to talk about the issues caused by non-compliant software and why more features are not always good, especially in the case of the upgrade of tools on a single platform.

Why I Shifted From Arch Linux to Debian Linux?

Preface: The following article is based on my personal experience with Arch Linux and Debian Linux. While I appreciate both distributions for their unique strengths and different use cases, the information provided reflects my own opinions and experiences. Your experiences may vary. What was I going for Initially? Bit of my Story as a Beginner Linux User When I started using Linux, I was going through security stuff and learning computer security when I was in my High School.

Tmux is the Ultimate Choice for Power Users - An Awesome Terminal Multiplexer for Managing Persistent Sessions

What is Tmux? Tmux is a Terminal Multiplexer Application for Linux and MacOS for managing terminal sessions and Windows. It is to be mentioned that Tmux is not a terminal emulator, instead, it’s a terminal application, a binary that allows you to stay productive over your terminal. It doesn’t matter which terminal emulator you are using (although I recommend the Suckless Terminal). The functionality of managing Windows and the session doesn’t happen on the desktop GUI side but on the terminal session, you are working with.

The Concept of RSS Feed - A Reliable Way for Publishers and Subscribers Model

Preface: I have been looking for simple solutions in nearly everything related to computing. A lot of software designed these days is not designed per your requirements but as per companies’ profit. Hence, most of these solutions end up sucking your time and energy and need to be replaced by more optimised solutions that would boost your productivity in the right way. RSS Feed is something that I found after searching for solutions related to subscribing to websites or creators without sucking much resources and design that is reliable to the user itself.

Breaking RSA Encryption on Hardware Devices with Side Channel Power Analysis — Leaking the Private Key by Exploiting Square-Multiply Algorithm

Preface: This article is about leaking the private key from hardware devices that implement RSA encryption which is part of hardware hacking. The author is not responsible for any damage caused by the given information. It is recommended to be careful while performing these attacks as they can damage the hardware or even destroy it. All the information provided here is for educational purposes. There are no such prerequisites for understanding the theory, although knowledge about modular arithmetic, basics of encryption mathematics, basic electronics, etc.

Serious Reconnaissance with Unmanned Aerial Vehicles — Mapping Out Devices in an Area with Drones

Introduction: The following content is for educational purposes and for hackers living in basements knowing enough ethics. The author is not responsible for any damage caused by the knowledge provided here and does not support anything like that. It’s essential to check out the rules in the area of reconnaissance about the tactics provided here and the use of materials (unless and until there is any apocalypse and a solid recon is required).

The Fundamentals of Hardware Hacking — Breaking and Reverse Engineering Smart IoT Devices

Disclaimer — This is an introductory article about Hardware Hacking and Security of IoT Devices. None of the mentioned information or techniques are intended for any illegal purposes and the author is not responsible for any damage. It’s advisable to experiment on devices that you own or have explicit permission to do so. Rest of all, hardware hacking is fun! The Beauty of Electronic Devices In the ever-growing world of smart devices and the connectivity of things to the internet, life has become more convenient than ever.

Setting Up a Remote Git Server — A Simple and Concise Step-by-Step Guide to Host a Private Git Server

Preface: This is a concise and simple guide to hosting a remote git server. I have been researching this topic for a while and came up with the idea of writing an article with a step-by-step guide for hosting a private git server. Covering all the aspects of git is not possible in a single article, so it’s assumed that the reader has prior knowledge of git and version control.

The Nature of Linux Kernel Development — Difference Between Rules of Kernel Level and User-Space Application Level

Preface: This article is intended to explain a clear distinction between the core principles of Linux Kernel Development and User-Application Level Development. The provided information is based on my research on Kernel Development through various sources and I have tried to make it as accurate as possible. Efforts have been made to explain it as simply and concisely as possible. Introduction to the Nature of Linux Kernel Linux Kernel is the abstraction layer between the Operating System and the Hardware in the system.

Linux Process Scheduling — The Reason your Linux System Processes so Efficiently (Kernel Perspective)

Preface: I was going through the book “Linux Kernel Development” by Robert Love, one of the best books I have referred to for low-level stuff and understanding the workings of Linux. I study this book with intensity, simplify the concepts and write it down here so that the readers can get a straightforward description and all they need to know about the topic. Covering the whole Linux Process Scheduling is not possible and is not the goal of this article.

Linux Processes — A Kernel’s Perspective Explained with Clarity and Simplicity

Preface: I have been going through the book “Linux Kernel Development” by Robert Love which I highly recommend for understanding the Linux Kernel in depth. I decided to write this article to explain “Linux Processes” simply and concisely. The topic itself is broad and is not explained into the deepest of it, but essential for Linux Administrators, Developers and even Linux users to appreciate the beauty of the Kernel they make use of every day.

NGINX for Deploying Next.js Application on AWS EC2 with AWS ELB — Control and Stability of Deployments

I was looking for an article like this a few days ago, which I didn’t find at that time, so I did the deployment on my own and came up with this article to prevent other developers from saving those efforts and focusing on development. I am not explaining every single step and have provided links for references. I prefer manual deployment of applications over automated (and even serverless) methods. Although they are convenient and require less effort by the developers, they are bound to the providers and offer less control over the underlying system.

Configuring and Building the Linux Kernel — Absolute Guide to Compiling Your Kernel

Linux Kernel is an Open-Source Software and the user is free to modify and customise it as per the requirements. The modification of the Kernel requires a deep understanding of the working of the Kernel, although patches are available to make the Kernel optimised for specific hardware. Linux Kernel Source Code has various options to configure the drivers, modules, preferences on hardware options, etc. This part can be studied by the user and is pretty much easy to work with.

The Elegance of the Linux Kernel — A Concise History of Unix and the Creation of the Linux Kernel

Introduction and Context I was going through the book “Linux Kernel Development” by Robert Love, an absolute guide to getting started with Linux Kernel Development and a highly recommended book for understanding the core of the Linux Kernel. Linux Kernel has been one of the most important software ever written and is even considered one of the biggest projects ever undertaken by a single person. The idea of the Linux Kernel was initiated by Linus Torvalds, a student from the University of Helsinki and is maintained by him to date (while writing this article).

Linux Shell Scripting — A Suckless and Concise Guide to the Command-line of Linux

Prior Statements: This is a concise guide on the Linux Shell Scripting while consolidating all the facts about the Linux Shell for quick developer’s reference while using Linux. I am referencing the Bash (Bourne-Again Shell) which is the default shell for Linux-based systems. I will also be providing references and external links to dive into depth and not fill the article with too much explanation about a single topic which is not universally required by all the readers.

Suckless Utilities for Arch Linux — The Most Minimal Way Run a Computer

Suckless utilities have been my favourite at this time and kind of essentials for my use of the computer. I have been using Arch Linux for a fair amount of time now and I started using it with XFCE for few weeks. I would appreciate the XFCE desktop environment for it’s smoothness and light-weight nature with works really fine when newly shifting on Arch Linux. But then I learnt the suckless ecosystem and eventually shifted to it as my full-time environment.

Installing Pacman in Arch Linux — When You Blow it Up

Let me suckless and divide the article into two parts: My story how I blew up Pacman Package Manager How to reinstall the Pacman Package Manager If you only care about the second part, skip the first one. The Scenario — Blow it Up I was trying to install the pacman game from the Internet to get it running on my Arch Linux Terminal (I use Suckless Terminal BTW). When I got it installed and played it, it was super awesome.

Boot Process of Computers — A Learner’s Perspective Of Exploring the Depth of Computers

Prior Clarifications: Here, I will be providing a philosophical explanation about the bootloaders and understanding them in a simple and as minimal way as possible. This is not supposed to be a manual for bootloader or provide any advice for experimenting over your live system. It’s my journey to understand computers (one of the most complex creations of mankind) and I will be stating my thoughts. Take it with a pinch of salt.

Networking Fundamentals for Linux Administrators — A Suckless and Concise Explanation

Statistics are clear on the fact that 96.3% (while writing this article) of the servers use Linux as their Operating System which is no doubt what every other Linux user on this Earth expects. I believe that the Linux Administrator has to take the shot about the configuration of Networking in Linux Based Server. Some of the underlying concepts remain the same for any other distros but it is mainly intended for Linux.

Operating Systems and Low-Level Access to the Hardware — Why should you learn it?

Today, I completed the whole read of the book “Linux Kernel in a Nutshell” by Greg Kroah-Hartman and I highly recommend that you go through it if you want to understand how to build your custom configuration of Linux Kernel and all you need to know about all the nuts and bolts. It’s always great to have such handbooks around the desk. This blog is about why it’s so awesome to look into the operating system you are using with your hardware and why have a grasp on the Low-Level aspects of a computer.

Arch Linux Custom Builds — Freedom of the Operating System

While writing this blog, I was reading the book “Linux Kernel in a Nutshell” by Greg Kroab-Hartman and as far as the pages of the book are concerned, it seems to be a two-day read (this is a handbook so reading it once and having around the desk is super useful). By the way, the author has the book left open for download http://www.kroah.com/lkn/ so check that out if you want to follow up.

Bypassing the Linux Login to access the files (with Physical Access), even the root!

Imagine being away from the computer for a couple of minutes and getting to know that the system has been compromised and a backdoor has been installed into the system. “The system was locked?” doesn’t matter, without the bios security implementation (which most probably would not be implemented), all the files can be recovered without any login made to the Login Screen. This goes with the story of me trying to get my Wi-Fi troubleshooting in Arch Linux where I was trying to upgrade the Kernel of my System to get the Wi-Fi working properly (as mentioned in the previous blog.

The Nature of Linux Kernel Development — Difference Between Rules of Kernel Level and User-Space Application Level

Planted February 4, 2024

Preface: This article is intended to explain a clear distinction between the core principles of Linux Kernel Development and User-Application Level Development. The provided information is based on my research on Kernel Development through various sources and I have tried to make it as accurate as possible. Efforts have been made to explain it as simply and concisely as possible.

Introduction to the Nature of Linux Kernel

Linux Kernel is the abstraction layer between the Operating System and the Hardware in the system. While most User-Application Developers are familiar with standard ways of application development, they reside in the comfort of the Operating System and often underestimate the features provided by it. Since the Linux Kernel goes deep down to the low level, it does not have such support for convenience. There is no definition of those aspects! For example, usually, printf would be used for printing messages generated by the software. These are usually built-in functions and need to be imported from some external dependencies. But in the case of Kernel, there are no external dependencies! or even the definition of it. Since printf prints messages in the console (your terminal), there is no console-like thing in a bare metal system. Hence, development at such a low level is different from the usual application-level development. It’s not difficult or tedious, it’s different. I would like to present a perspective on this in the next section.

The Perspective of the Author in this Topic

Now these are my very personal opinions and may not be accurate but are my best intentions.

When developers learn to program, usually they learn while being in the comfort of the underlying Operating System and a Kernel that is residing under it to do its job. It’s not always required by application-level developers to go through a lot of low-level and is usually good for them to improve their application-level skills to make progress on that side of computers. On the other hand, it confines their ability to get out of the zone and appreciate the working of such low-level programs that make their life easier. Programming at such a low level does not incorporate external dependencies and hence, it should be noted that programming is not just about getting the work done with external dependencies. The external dependencies are not the real syntax of it. Writing printf when in need of console output is convenient but not the syntax of the programming language. When going down deep into the roots, this convenience decreases and it’s time to understand the real “syntax”. The syntax is the same for the language, but the implementation changes. Now it’s not just some user-friendly output, it would be defining how the underlying hardware functions.

While understanding Kernel Development, I always like to think of C Language as its Assembly code. This gives more closeness to the processor and therefore understanding things more from the firmware development perspective. Since the process understands assembly and this is how the underlying core of any application works, this gives more convenience while imagining the step-by-step execution of the code.

The more you go deep, the more you get out of the illusion of development and understand the real rules of computers.

The Seven Important Points in Kernel Development

Here are the absolutely important points about the Kernel Development:

  1. The Kernel has access to neither the C library nor the standard C headers. There is no definition of external dependencies while programming at such low levels.
  2. The Kernel is developed in the GNU C Language
  3. The Kernel lacks the memory protection afforded to user-space (So developers who got used to application-level programming need to get out of the illusion of security convenience).
  4. The Kernel Cannot easily execute floating-point operations.
  5. The Kernel has a small per-process fixed-size stack.
  6. Because the Kernel has asynchronous interrupts, is preemptive, and supports SMP, synchronization and concurrency are major concerns within the Kernel.
  7. Portability of important (It needs to run on a variety of CPU Architectures).

No libc or Standard Headers

A Header file contains functions that are predefined to make the development more convenient as they contain utilities that are complex and reusable. So developing them once and using them makes more sense. They are just external dependencies which are installed on the system. But in the case of Kernel, there can’t be any external dependencies on the system (as there is no system, it’s just a circuit board so so-called motherboard in the case of computers). Hence, the Kernel needs to have it’s all its code consolidated in it and defined in it. The Kernel has header files which are used in cases of reusability of code and standard functions but they have been developed for their purpose and hence, are required to be studied differently. For example, there is no printf in the Kernel as there is no console, but there is a function called printk that prints the Kernel-generated messages (you can use the dmesg command in your Linux system to print the Kernel-generated messages on the console). The console is a concept that needs to be developed while writing the Kernel and is not something that exists already. Hence, printk is defined as printing the message in a way that it would be on the console that is defined to show on the monitor.

However, there are libc libraries included in the Linux Kernel for convenience of development which would be essential and reusable in a lot of parts of the code. For example, common string manipulation is included in the lib/string.c which can be included by using linux/string.h.

The Use of GNU C Programming Language

Here is the part where the Kernel Development deviates from usual Application-Level Development. Usually, ANSI C is used at the Application Level while the Linux Kernel is developed in the GNU C Programming Language. The Kernel can be compiled with gcc and has also got the support of the Intel compiler. The usage of GNU C is due to the support provided by it for various functions that are essential for firmware development. Some of the essential extensions are explained here:

  1. Inline Functions: Both C99 and GNU C support inline functions. These functions are inserted inline into each function call site. This eliminated this overhead of function invocation and return (register to save and restore) and allows for potentially greater optimisations as the compiler can optimise both the caller and the function as one.
  2. Inline Assembly: The gcc compiler enables the embedding of assembly instructions in otherwise normal C functions. This is used in the parts where it is unique to the system Architecture and needs particular adjustments.
  3. Branch Annotation: The gcc compiler has a built-in directive that optimises conditional branches as either very likely taken or very unlikely taken. The Kernel wraps the directive in easy-to-use macros, likely() and unlikely().

No Memory Protection

The concept of memory protection comes in the Application-Level Development for secure coding practices. Languages like Python manage their memory with built-in functions whereas languages like Rust (one of my favourite languages while writing this article) use mechanisms like borrow-checker to ensure secure memory management. But in the case of Kernel Development, there is just a piece of hardware which have memory free to access! Hence, to provide memory management to make sure that the memory is used efficiently and securely by the operating system (actually this is where the Kernel Exploits happen due to overflow in cases where memory management logic is compromised). Hence, the Kernel has to impose some rules for itself to control the usage of the memory and prevent unwanted behaviours.

No (Easy) use of Floating Point

When the user-space process uses the floating-point instructions, the Kernel manages the transition from integer to floating-point mode. The Kernel handles the floating-point instructions differently on each architecture but usually catches a trap and then initiates the transition from integer to floating-point mode.

But the Kernel does not enjoy the luxury of catching a trap (it can’t easily trap itself) and hence, manual procedures need to be used. This is tedious and hence, usually not preferred and is usually used only if there is a strict requirement.

Small, Fixed-Size Stack

User-Space Applications can make use of as many variables as they want due to the large stack that can dynamically grow. But it’s not the case with Kernel Development. The Kernel Stack is neither large nor dynamic; it is small and fixed in size. It’s 8KB for 32-bit architecture and 16KB for 64-bit architecture which is fixed and absolute. Each process receives its stack.

Synchronization and Concurrency

The kernel is susceptible to race conditions. Shared and concurrent access to resources is allowed and hence, synchronization is required to prevent race conditions.

  1. Linux is a preemptive (Kernel can stop any process as per the allocated timeslice, more explained in another article) multitasking operating system. Hence, the Kernel needs to synchronise between the switching tasks.
  2. Linux supports symmetrical multiprocessing (SMP). Therefore, without proper protection, Kernel code executing simultaneously on two or more processors can concurrently access the same resource.
  3. Interrupts occur asynchronously with respect to the currently executing code. Therefore, without proper protection, an interrupt can occur in the midst of accessing a resource, and the interrupt handler can then access the same resource.
  4. Since the Linux Kernel is preemptive, without protection, Kernel code can be preempted in favour of different code that then accesses the same resource.

Portability of the Kernel

The Linux Kernel needs to be portable due to its wide range of applications. Linux Kernel powers large servers and infrastructures as well as embedded devices. Hence, hardware considerations have to be taken care of and must be independent of the platform it runs on. This is something that is at the heart of the Linux Kernel, to run on a wide range of systems.

Conclusion

Kernel development is a wide topic and has a very different nature than application-level development. Although it has a very different nature, it’s not difficult. Understanding and exploring it gives a deeper dive into the low levels of the computers and helps understand a lot about the real nature of the hardware of the computers. It’s just the fact that most developers have got so caught up with User-Space Application development rules that they feel low levels as rigorous and tedious, which is not the case. It’s just how the real under-the-hood things happen, this is how it’s governed.