Deep Dive Articles

The Best Way to Learn a New Programming Language from Scratch - How I Approach and Learn Any Programming Language Efficiently

Preface: The following article is based on my experiences and opinions on learning programming languages. I have been learning about computers in-depth and find learning languages with specific purposes useful. For example, Python was for beginning, C is for memory management and Rust is for learning to write memory-safe code in my case. This time, I started learning Golang which is known for its simple syntax and powerful performance, and widely known for its awesome concurrency.

Why More is Not Always Good in Terms of Software - Words on Cross Platform Utilities, Bash-ism, and POSIX Compliance.

Preface: This article is about my views on software compliance and cross-platform support, and reflects my opinions and experience with the subject. Your experience and opinions may vary, which I respect. What I am Specifically Talking About? I am going to talk about the issues caused by non-compliant software and why more features are not always good, especially in the case of the upgrade of tools on a single platform.

Why I Shifted From Arch Linux to Debian Linux?

Preface: The following article is based on my personal experience with Arch Linux and Debian Linux. While I appreciate both distributions for their unique strengths and different use cases, the information provided reflects my own opinions and experiences. Your experiences may vary. What was I going for Initially? Bit of my Story as a Beginner Linux User When I started using Linux, I was going through security stuff and learning computer security when I was in my High School.

Tmux is the Ultimate Choice for Power Users - An Awesome Terminal Multiplexer for Managing Persistent Sessions

What is Tmux? Tmux is a Terminal Multiplexer Application for Linux and MacOS for managing terminal sessions and Windows. It is to be mentioned that Tmux is not a terminal emulator, instead, it’s a terminal application, a binary that allows you to stay productive over your terminal. It doesn’t matter which terminal emulator you are using (although I recommend the Suckless Terminal). The functionality of managing Windows and the session doesn’t happen on the desktop GUI side but on the terminal session, you are working with.

The Concept of RSS Feed - A Reliable Way for Publishers and Subscribers Model

Preface: I have been looking for simple solutions in nearly everything related to computing. A lot of software designed these days is not designed per your requirements but as per companies’ profit. Hence, most of these solutions end up sucking your time and energy and need to be replaced by more optimised solutions that would boost your productivity in the right way. RSS Feed is something that I found after searching for solutions related to subscribing to websites or creators without sucking much resources and design that is reliable to the user itself.

Breaking RSA Encryption on Hardware Devices with Side Channel Power Analysis — Leaking the Private Key by Exploiting Square-Multiply Algorithm

Preface: This article is about leaking the private key from hardware devices that implement RSA encryption which is part of hardware hacking. The author is not responsible for any damage caused by the given information. It is recommended to be careful while performing these attacks as they can damage the hardware or even destroy it. All the information provided here is for educational purposes. There are no such prerequisites for understanding the theory, although knowledge about modular arithmetic, basics of encryption mathematics, basic electronics, etc.

Serious Reconnaissance with Unmanned Aerial Vehicles — Mapping Out Devices in an Area with Drones

Introduction: The following content is for educational purposes and for hackers living in basements knowing enough ethics. The author is not responsible for any damage caused by the knowledge provided here and does not support anything like that. It’s essential to check out the rules in the area of reconnaissance about the tactics provided here and the use of materials (unless and until there is any apocalypse and a solid recon is required).

The Fundamentals of Hardware Hacking — Breaking and Reverse Engineering Smart IoT Devices

Disclaimer — This is an introductory article about Hardware Hacking and Security of IoT Devices. None of the mentioned information or techniques are intended for any illegal purposes and the author is not responsible for any damage. It’s advisable to experiment on devices that you own or have explicit permission to do so. Rest of all, hardware hacking is fun! The Beauty of Electronic Devices In the ever-growing world of smart devices and the connectivity of things to the internet, life has become more convenient than ever.

Setting Up a Remote Git Server — A Simple and Concise Step-by-Step Guide to Host a Private Git Server

Preface: This is a concise and simple guide to hosting a remote git server. I have been researching this topic for a while and came up with the idea of writing an article with a step-by-step guide for hosting a private git server. Covering all the aspects of git is not possible in a single article, so it’s assumed that the reader has prior knowledge of git and version control.

The Nature of Linux Kernel Development — Difference Between Rules of Kernel Level and User-Space Application Level

Preface: This article is intended to explain a clear distinction between the core principles of Linux Kernel Development and User-Application Level Development. The provided information is based on my research on Kernel Development through various sources and I have tried to make it as accurate as possible. Efforts have been made to explain it as simply and concisely as possible. Introduction to the Nature of Linux Kernel Linux Kernel is the abstraction layer between the Operating System and the Hardware in the system.

Linux Process Scheduling — The Reason your Linux System Processes so Efficiently (Kernel Perspective)

Preface: I was going through the book “Linux Kernel Development” by Robert Love, one of the best books I have referred to for low-level stuff and understanding the workings of Linux. I study this book with intensity, simplify the concepts and write it down here so that the readers can get a straightforward description and all they need to know about the topic. Covering the whole Linux Process Scheduling is not possible and is not the goal of this article.

Linux Processes — A Kernel’s Perspective Explained with Clarity and Simplicity

Preface: I have been going through the book “Linux Kernel Development” by Robert Love which I highly recommend for understanding the Linux Kernel in depth. I decided to write this article to explain “Linux Processes” simply and concisely. The topic itself is broad and is not explained into the deepest of it, but essential for Linux Administrators, Developers and even Linux users to appreciate the beauty of the Kernel they make use of every day.

NGINX for Deploying Next.js Application on AWS EC2 with AWS ELB — Control and Stability of Deployments

I was looking for an article like this a few days ago, which I didn’t find at that time, so I did the deployment on my own and came up with this article to prevent other developers from saving those efforts and focusing on development. I am not explaining every single step and have provided links for references. I prefer manual deployment of applications over automated (and even serverless) methods. Although they are convenient and require less effort by the developers, they are bound to the providers and offer less control over the underlying system.

Configuring and Building the Linux Kernel — Absolute Guide to Compiling Your Kernel

Linux Kernel is an Open-Source Software and the user is free to modify and customise it as per the requirements. The modification of the Kernel requires a deep understanding of the working of the Kernel, although patches are available to make the Kernel optimised for specific hardware. Linux Kernel Source Code has various options to configure the drivers, modules, preferences on hardware options, etc. This part can be studied by the user and is pretty much easy to work with.

The Elegance of the Linux Kernel — A Concise History of Unix and the Creation of the Linux Kernel

Introduction and Context I was going through the book “Linux Kernel Development” by Robert Love, an absolute guide to getting started with Linux Kernel Development and a highly recommended book for understanding the core of the Linux Kernel. Linux Kernel has been one of the most important software ever written and is even considered one of the biggest projects ever undertaken by a single person. The idea of the Linux Kernel was initiated by Linus Torvalds, a student from the University of Helsinki and is maintained by him to date (while writing this article).

Linux Shell Scripting — A Suckless and Concise Guide to the Command-line of Linux

Prior Statements: This is a concise guide on the Linux Shell Scripting while consolidating all the facts about the Linux Shell for quick developer’s reference while using Linux. I am referencing the Bash (Bourne-Again Shell) which is the default shell for Linux-based systems. I will also be providing references and external links to dive into depth and not fill the article with too much explanation about a single topic which is not universally required by all the readers.

Suckless Utilities for Arch Linux — The Most Minimal Way Run a Computer

Suckless utilities have been my favourite at this time and kind of essentials for my use of the computer. I have been using Arch Linux for a fair amount of time now and I started using it with XFCE for few weeks. I would appreciate the XFCE desktop environment for it’s smoothness and light-weight nature with works really fine when newly shifting on Arch Linux. But then I learnt the suckless ecosystem and eventually shifted to it as my full-time environment.

Installing Pacman in Arch Linux — When You Blow it Up

Let me suckless and divide the article into two parts: My story how I blew up Pacman Package Manager How to reinstall the Pacman Package Manager If you only care about the second part, skip the first one. The Scenario — Blow it Up I was trying to install the pacman game from the Internet to get it running on my Arch Linux Terminal (I use Suckless Terminal BTW). When I got it installed and played it, it was super awesome.

Boot Process of Computers — A Learner’s Perspective Of Exploring the Depth of Computers

Prior Clarifications: Here, I will be providing a philosophical explanation about the bootloaders and understanding them in a simple and as minimal way as possible. This is not supposed to be a manual for bootloader or provide any advice for experimenting over your live system. It’s my journey to understand computers (one of the most complex creations of mankind) and I will be stating my thoughts. Take it with a pinch of salt.

Networking Fundamentals for Linux Administrators — A Suckless and Concise Explanation

Statistics are clear on the fact that 96.3% (while writing this article) of the servers use Linux as their Operating System which is no doubt what every other Linux user on this Earth expects. I believe that the Linux Administrator has to take the shot about the configuration of Networking in Linux Based Server. Some of the underlying concepts remain the same for any other distros but it is mainly intended for Linux.

Operating Systems and Low-Level Access to the Hardware — Why should you learn it?

Today, I completed the whole read of the book “Linux Kernel in a Nutshell” by Greg Kroah-Hartman and I highly recommend that you go through it if you want to understand how to build your custom configuration of Linux Kernel and all you need to know about all the nuts and bolts. It’s always great to have such handbooks around the desk. This blog is about why it’s so awesome to look into the operating system you are using with your hardware and why have a grasp on the Low-Level aspects of a computer.

Arch Linux Custom Builds — Freedom of the Operating System

While writing this blog, I was reading the book “Linux Kernel in a Nutshell” by Greg Kroab-Hartman and as far as the pages of the book are concerned, it seems to be a two-day read (this is a handbook so reading it once and having around the desk is super useful). By the way, the author has the book left open for download http://www.kroah.com/lkn/ so check that out if you want to follow up.

Bypassing the Linux Login to access the files (with Physical Access), even the root!

Imagine being away from the computer for a couple of minutes and getting to know that the system has been compromised and a backdoor has been installed into the system. “The system was locked?” doesn’t matter, without the bios security implementation (which most probably would not be implemented), all the files can be recovered without any login made to the Login Screen. This goes with the story of me trying to get my Wi-Fi troubleshooting in Arch Linux where I was trying to upgrade the Kernel of my System to get the Wi-Fi working properly (as mentioned in the previous blog.

Linux Process Scheduling — The Reason your Linux System Processes so Efficiently (Kernel Perspective)

Planted February 2, 2024

Preface: I was going through the book “Linux Kernel Development” by Robert Love, one of the best books I have referred to for low-level stuff and understanding the workings of Linux. I study this book with intensity, simplify the concepts and write it down here so that the readers can get a straightforward description and all they need to know about the topic. Covering the whole Linux Process Scheduling is not possible and is not the goal of this article. It’s to let you know the topic in more depth than a summary can provide and create a pathway you need to go to learn the topic.

A Quick Introduction to the Topic

The handling of processes in an Operating System like Linux is a vital topic for understanding the working of the Kernel. The logic behind scheduling processes in the system and prioritising them in a manner that is the most optimised and efficient, while considering some trade-off gives an insight on real-life engineering behind low-level computer stuff.

To understand what a process is, I highly recommend you go through my previous article, “Linux Processes — A Kernel’s Perspective Explained with Clarity and Simplicity” where I explain the processes and threads simply and concisely.

To understand the topic, appreciating the concept and then moving towards understanding the code structure is important. Here, the code part would not be covered as explaining the chain of concepts is taken to the priority.

The Significance of Process Scheduling

A Kernel by definition is the abstraction layer between the Operating System and the Hardware. An Operating System has a lot of tasks running simultaneously to make things happen in the way they are intended to be. Even an idle system has a lot going under the hood and the Kernel needs to interact with the hardware in a way that each task is carried out in the most appropriate way possible.

Since an Operating Systems have multiple processes running (definitely more than the processors on the system), the Kernel has to decide which process to consider at the given time. Modern hardware has multiple processors (multiple cores) and the Kernel needs to make use of them in the most reliable way possible. This whole work of decision-making is made by the process scheduler in the Linux Kernel.

diagram

If there are any processes that too in a runnable state, the processor should be always running. In the case of multiple, time-consuming processes, some processes must be paused or kept waiting in a way that the system functions in the way it is intended to work. This happens with the mechanism of prioritisation of tasks, which the process scheduler in Linux Kernel handles.

The Concept of Multitasking

A Multitasking Operating System by definition has the ability to execute multiple processes on the system simultaneously. On a single processor machine, it can give an illusion that all the processes are running simultaneously at the same time (actually the processor can focus on one process at any time, but the execution is not only the definition of the process, it is the whole thing from memory allocation and multiple processes spawning. Hence, the system seems to be having multiple processes running at the same time. A task which is marked as sleeping is also a process, it’s just the processor is busy with something else). On a multiprocessor machine, these processes are running simultaneously in parallel while some more processes are blocked or sleeping, waiting to get executed when a processor is free. A lot of processes might be residing in the memory, they might just be waiting for some input from an external device, and at that time, it might also be the case that only one process is in a runnable state. The Linux Scheduler manages all this with precise mechanisms to make the system smooth and interactive.

There are two types of multitasking in operating systems: cooperative multitasking and preemptive multitasking. A very concise definition of them is that cooperative multitasking operating systems have no right to stop the process during the execution and the process has to exit on its own. On the other hand, a preemptive multitasking operating system has the right to stop the execution of the process at any time. Unix, at its core, has been implementing preemptive multitasking and Linux Kernel has been implementing preemptive multitasking (it’s obvious that if cooperative multitasking is used and a process hangs and does not exit due to any reason, it can potentially bring down the whole system).

The Implementation of Policy Policy is the behaviour of the scheduler that determines what runs when. To optimally use the processor time, Linux Process Scheduler implements some policy that determines which process to run at a given time and what process to keep waiting to make the most out of system resources.

There are a few terms that are considered while following a policy. They are:

  1. I/O-Bound Versus Processor-Bound Processes
  2. Process Priority
  3. Timeslice

I/O-Bound Versus Processor-Bound Processes

Processes can be divided into two types, the processes which are controlled by external devices (like keyboard, mouse, drives, etc.) are I/O-Bound Processes. These processes are most of the time idle and consume less processor time. But these are the ones that are responsible for making the system responsive to the user. For example, keyboard input must be processed at each stroke or else the system will not feel responsive. No matter how fast the user types (it’s not going to be that fast for the computer), this process is idle most of the time and resides in the memory. Hence, this process takes less processor time but needs to run as soon as it needs to be. On the other hand, Processor-Bound Processes consume significantly more processor time but do not need to be processed at a particular stroke of time. They can be kept in the memory and processed as per the processor’s state. For example, video rendering takes significant processor time but does not care the amount of time is taken to render the whole video (of course fast rendering would be great but a small latency in video rendering can be accepted than latency in typing in a text editor).

The Process Scheduler is responsible for deciding when to process an I/O-Bound Process and when to pause/resume the execution of a Processor-Bound Process.

Note that a process can be both, an I/O-Bound Process and a Processor-Bound Process. For example, Word-Processor, which requires quick processing of keyboard input and also does a spell-check simultaneously or some macro calculations).

Process Priority

Setting up a priority value for a process helps the Linux Scheduler to measure the weight of the process and its requirements to the system. A term called nice value is used which ranges from -20 to 19. A lower nice value corresponds to higher priority and vice versa (a less nice value means the process is less nice to other processes and requires more processor attention).

A very important point to be mentioned here: Unix allocates timeslices proportionally and not absolutely unlike some other operating systems and the same goes with Linux. This means that having a lower nice value (higher priority) will give a process relatively more time than a process with a higher nice value (lower priority). This depends upon the number of processes in the system and certain parameters in the Operating System. More discussion on this topic is out of the scope as this is a huge topic about why this is implemented and the approach towards perfect multitasking.

A second range is real-time priority which is on a different track with nice value. Opposite from nice value, higher real-time priority refers to more priority and vice versa. Linux implements real-time priority as per Unix standards specifically POSIX.1b standards. All real-time processes have higher priority than the normal processes.

Timeslice

A timeslice is a numerical value that represents how long a task can run until it is preempted. The policy of the scheduler dictates the default timeslice. Too long timeslice reduce the responsiveness of the system as more processor time will be allocated to individual processes, reducing the speed of the queue of processes. If too little timeslice is provided, then the switching will happen so fast that the process of switching itself will take most of the time (this is called the switching overhead and occurs when a process is preempted and switched to the next one consuming some time) which further decreases the throughput of the processor (the processor wastes time in switching more than it processes). Hence, an optimal value is set to overcome this.

Concluding with More to Explore

The timeslice allocation is done by the process scheduler in a relative way proportional to the number of processes running in the system. This has a bit deeper explanation of why this has been implemented and why this optimises the resource allocation. Explaining that would take another article and is out of the scope of this one.

Understanding Process Scheduling in the Linux Kernel is an important path to understanding the working and philosophies of it.