Archives 2019

Linux Boot Series – SystemD

boot process SystemD

Though SystemD will be compared to its predecessor SystemV for a long time , it has much more to offer in terms of System Management. It is a new way how Linux interact with the underline Objects such as hardware , sockets , application processes and many more.

Understand How SystemD works

systemd is a system and service manager for Linux, compatible with SysV and LSB init scripts.

Features
  1. Contrary to its predecessor SystemD handles processes parallel
  2. socket and D-Bus activation
    traditionally services will be configured to start on boot, but with systemd it is more event driven and you can configure to start when something connects to a specific port or a device get connected , this is called socket and d-bus activation.
  3. Offers on-demand starting of daemons, also keeps track of processes using Linux cgroups
  4. Supports snapshot and restoring of the system state
  5. Maintains mount and automount points
  6. Implements an elaborate transactional dependency-based service control logic.

systemD working

Concept of Units in SystemD

SystemD manages units, which are representations of system resources and services.

Type of Units in SystemD
  1. Service
    A Service unit is used to manage service, the unit file include instruction to start , stop , restart the service.
  2. Socket A network socket associated with a service.
  3. Device
    Unit file related to Device is used to manage Device , start , stop , auto etc.
  4. Mount Unit manages the Mount points via SystemD
  5. Automount Unit file mounts the file system on system boot. This feature might replace traditional fstab files in the near future.
  6. Swap Mounts swap space on the system
  7. Target Targets are much like Runlevel used previously to manage different services to start and stop at different
  8. Path
    A path for path-based activation. For example, you can start services based on the state of a certain path, such as whether it exists or not.
  9. Timer Timer unit is used similar to Crontab to schedule other units.
  10. Snapshot
    A snapshot of the current systemd state. Usually used to rollback after making temporary changes to systemd.
  11. Slice Restriction of resources through Linux Control Group nodes (cgroups).
  12. Scope Information from systemd bus interfaces. Usually used to manage external system processes.
systemctlcommand

is the primary tool to manage SystemD. It can be used for starting , stopping of services as well as enabling and disabling , this was previously performed with service and chkconfig commands in the previous versions.

Basic Service Management Syntax
Description: Command
Start the Service systemctl start foo
Stop the Service systemctl stop foo
Restart the Service systemctl restart foo
Status of the Service systemctl status foo
Enable Service to start at bootime systemctl enable foo
Disable the Service systemctl disable foo
Check if service is enabled
?
systemctl is-enabled foo

Mask
the Service
systemctl mask foo
Reload the updated unit file systemctl daemon-reload
Show Failed Services systemctl -failed
Reset any failed service systemctl reset-failed
Show properties of the Unit systemctl show < service >
Edit the Service Unit systemctl edit < service >
Edit the Full Service Unit systemctl edit --full < service >
Run on remote host systemctl -H < host_name > status network
Changing System State
Reboot host systemctl reboot
Poweroff host systemctl poweroff
Switch to Emergency mode systemctl emergency
Log back to default mode (Multi-User) systemctl default
Viewing Log Messages
Show all log messages journalctl
Show only kernel log messages journalctl -k
Show log for specific service journalctl -u network.service
Follow messages as they appear journalctl -f

Besides services, most systemd commands can work with these unit types: paths,
slices, snapshots, sockets, swaps, targets, and timers

Linux Boot Series – Part 2

linux boot process 2

Once the Hardware check POST “Power On Self Test” is completed and boot devices are identified , the last step by UEFI / BIOS was to identify the first boot device read the  <abbr title=”Master boot record”>MBR</abbr>. Its a 512 byte in the storage device that stores boot loader information, this can be considered as an index location that refer to other sectors for loading the Operating System. In most of the Linux distributions GRUB V2 is used as a bootloader as of this writeup.

GRUB V2 stands for “Grand Unified Bootloader, version 2” , it is the program that identifies and loads system kernel. At this point we should be clear why we use GRUB2 / GRUB V2 rather then just simply calling it GRUB ? that is because GRUB V2 is the rewrite of a legacy bootloader GRUB2 with many new features and a modular design. It is designed for multi-OS boot running multiple Linux, Unix and other proprietary Operating systems such as MS Windows. It can even identify multiple kernel for the Same Linux distribution and allow to boot from older version if required.

The default configuration file is
Ubuntu /boot/grub/grub.conf
RHEL 7 /boot/grub/grub.conf

Grub in itself is a complete topic to be discussed along with its configuration and management options which is discussed in GRUB section.

Once the Kernel is selected, Kernel along with initramfs is loaded in main memory and root file system gets mounted, the first process in legacy SystemV was the init process which will initiate the OS processes , but this has changed with Canonical Upstart and more recently SystemD. Both these systems have been designed to overcome what was previously the shortcomings of SystemV INIT system, both of these have comparatively similar features but do differ in design and architecture. As of now SystemD seems to lead with many big distribution such as Redhat , Fedora , CentOS , Debian and last but not least Ubuntu have given up Upstart in favor of SystemD, one of the reason was both of these system were causing more confusion for the software developers community.

To keep things simple i have divided the working of these three system in separate links below.

  1. System V Init (Legacy)
  2. Upstart
  3. SystemD

manage docker images

docker add remove

Once you have installed docker its time to manage , pull , remove docker images.

What is an Image

Image is a file that comprise of multiple layers used to execute code in a docker. It is built from the instruction for a complete and executable version of an application.

Note it does rely on the OS kernel

There are more then 1500 images created by different application teams such as by apache , ngnix , mysql , mariadb , these are called official docker images. One of the biggest repositories for these official docker images is

https://hub.docker.com

.

Installing Images

Downloading and installing an image on docker in Linux is a very straight forward way with a single command,example ngnix

docker pull <package_name>

To pull a particular version of an image

docker pull <package_name>:package_version_detail

by default pull will grab the latest version of the package and same goes for removing the version with docker rmi <package_name>

To see all the images that are downloaded on your machine use the command

docker images 

To delete an image you can type

docker rmi <package_name>:package_version_details

if no package version details are added it will delete the latest version of the package.

GRUB vs GRUB2

grub2

GRUB2 is a rewrite of legacy GRUB with many great updates, specially speed and security.

  • Configuration file has a new name
    grub.cfg replaced menu.lst , and configuration cannot be migrated from GRUB menu.lst to GRUB grub.cfg
  • New Commands
    Many new commands are added to GRUB2 which can be found in the link
  • grub.cfg gets automatically generatedThe file get automatically generated by grub-mkconfig This makes it easier to handle versioned kernel upgrades.
  • The configuration has updated syntax The syntax used in grub.cfg includes variables , conditionals, and loops something similar to scripting language.
  • Finding GRUB2 Kernel Files  GRUB2 has more reliable ways to find its own files and those of target kernels on multiple-disk systems
  • GRUB2 Multiple types  GRUB2 is available for multiple system such as PowerPC , PC BIOS, EFI , SPARC , MIFS
  • Many more file system supported GRUB2 supports many more file systems as compared to GRUB
  • GRUB2 reads  It can read files from LVM and RAID directly.
  • Graphical  GRUB2 has graphical terminal and graphical menu.
  • Modular design  GRUB 2 puts many facilities in dynamically loaded modules, allowing the core image to be smaller

Linux Boot Series – Part 1

linux boot process part 1

Boot process is one of the major part of troubleshooting an Operating System , it is the most critical time when administrators are tested to bring server up and running as soon as possible. Understanding how Operating system boot and what are the possible issues helps administrator to manage and configure Operating system which can not only boot faster but also recover and repaired in the fasted possible time.
The very first part of the boot process depends on the hardware architecture, there are few of them that are commonly used

  • Intel x86-based                         i386
  • AMD64 & Intel 64                      amd64
  • multiplatform for LPAE        generic-lpae
  • IBM POWER Systems             ppc64el
  • IBM z/Architecture                  s390x
BIOS based X86 Architecture

X86 systems are BIOS based and loads the first stage boot loader from the MBR of assigned storage , that inturn loads the boot loader stage 1.5 and 2 , default boot loader for linux is GRUB
UEFI-based x86 systems mount an EFI System Partition that contains a version of the GRUB boot loader. The EFI boot manager loads and runs GRUB as an EFI application.
Power Systems servers mount a PPC PReP partition that contains the Yaboot boot loader. The System Management Services (SMS) boot manager loads and runs yaboot.
IBM System z runs the z/IPL boot loader from a DASD or FCP-connected device that you specify when you IPL the partition that contains Linux Operating System
Note : BIOS and UEFI are both available in VMWare products as well as Oracle VirtualBox for latest configurations.

BIOS-based x86 Systems Details

BIOS (Basic input / output system) is a firmware interface in IBM compatible PCs and lately is also adopted by Virtual Software companies like VMWare and Virtualbox to be available in Virtual machines.
It is embedded on a chip in the motherboards for physical hardware and helps to scan and test all the devices in the system and selects the device to boot. Boot options is the list of devices in BIOS that provides list of bootable devices and the sequence to test bootable devices for the Operating system availability.
Usually, it checks any optical drives or USB storage devices present for bootable media, then, failing that, looks to the system’s hard drives. The BIOS then loads into memory whatever program is residing in the first sector of this device, called the Master Boot Record (MBR).
The MBR is only 512 bytes in size and contains machine code instructions for booting the machine, called a boot loader, along with the partition table. Once the BIOS finds and loads the boot loader program into memory, it gives control of the boot process to it.

UEFI based X86 Systems

UEFI is designed similar to BIOS with some great additions , unlike BIOS it run on its own architecture independ of the CPU and its own device driver. UEFI can mount partitions and read certain file systems. Although it has unique features its main tasks include searching for the bootable file system and passing on the control to the Operating system kernel. UEFI system identifies the partition with the GUID (globally unique identifier) which marks it as the EFI system partition. This partition contains applications compiled for the EFI architecture, which might include bootloaders for operating systems and utility software.
UEFI system includes an EFI boot manager that can boot the system from a default configuration or allow the user to choose from list of detected Operating systems. Once selected UEFI reads it into memory and gives control to the boot process.

super user setup in Ubuntu Debian distribution

super user

sudo Command is an important command in Linux user commands list. It is a safe way to execute privilege tasks , as well all commands executed with sudo are logged for audit purposes.

Running sudo command does not require root credentials , it is also possible to set minute details of sudo access in /etc/sudoers file.

Syntax

The /etc/sudoers file gets read in one pass , multiple entries might match but the last one has the highest precedence. It is advised to set the aliases before using them.
Comments can be inserted with # , with an exception that uid are also prefixed with an # symbol

Aliases

There are 4 type of aliases in sudoer file that can be used to assign permission

  • User Aliases
  • Runas Aliases
  • Command Aliases
  • Host Aliases

Aliases are the name for a user or group of users , host or group of hosts , a command or a group of commands.
Syntax: Alias_type NAME = value1,value2 ...

User Aliases
 # Everyone in the system group is covered under alias ADMINS
 User_Alias ADMINS = %admin
 # The users "tom", "james", are covered by the WEBDEV alias
 User_Alias WEBDEV = tom, james

In case you want to exclude a user or group of user from permission use !

 # This matches anybody in the USERS alias who isn't in WEBMASTERS or ADMINS aliases
 User_Alias LIMITED_USERS = USERS, !WEBMASTERS, !ADMINS
Runas Aliases

It is similar to User Alias except for the the fact it does allow user to be mentioned by UID’s this helps to match both usernames of a single UID as practiced in certain cases.
Basically UID’s are used for root user Runas_Alias ROOT = #0

# ROOT alias for uid 0 , Note #0 is not a comment
Runas_Alias ROOT = #0
#ADMINS alias for the group admin + user root
Runas_Alias ADMINS = %admin, root
Command Aliases

Command aliases are lists of commands and directories. You can use this to specify a group of commands. If you specify a directory it will include any file within that directory but not in any subdirectories.

# All the power options commands
 Cmnd_Alias POWER_CMDS = /sbin/poweroff, /sbin/reboot, /sbin/halt
# Admin commands
 Cmnd_Alias ADMIN_CMDS = /usr/sbin/passwd, 
# User Management Commands 
Cmnd_Alias USERMAN_CMDS = /usr/sbin/useradd, /usr/sbin/userdel, /usr/sbin/usermod, /usr/sbin/visudo
Host Aliases

A host alias is a list of hostname, ip address , networks , netgroups prefixed with a + plus symbol.
A host alias is a list of hostname, ip addresses, networks and netgroups (prefixed with a +).
Note: If you do not specify a netmask with a network the netmask of the hosts ethernet interface(s) will be used when matching, but it is a good practice to use netmask while configuring.

 # This is all the servers
 Host_Alias IAM_SERVERS = 10.10.2.5, 10.10.2.7, serverA
 # This is the whole network
 Host_Alias PUB_NET = 10.10.2.0/255.255.255.128
 # And this is every machine in the network that is not a server
 Host_Alias WORKSTATIONS = NETWORK, !SERVER
 # putting is all together
 # Host_Alias WORKSTATIONS = 10.10.2.0/255.255.255.128, !SERVERS
User Specifications

To make it all sense joining above declared aliases is the main part , this is where it is set WHO can run WHAT as WHO

  =   
 # LAMP Admins can run there commands provided they give password
 LAMPMIN LAMPSERVER= LAMP_CMDS
 # This lets run admin commands on all host under SERVER alias 
 ADMINS SERVERS= ADMIN_CMDS
 # This lets all the USERS run admin commands on the workstations provided 
# they give the root password or and admin password (using "sudo -u ")
 USERS WORKSTATIONS=(ADMINS) ADMIN_CMDS
 # This lets "patrick" run lamp commands without password on his local machine workstation10
 patrick workstation10= NOPASSWD: LAMP_CMDS
 # And this lets everybody print without requiring a password
 ALL ALL=(ALL) NOPASSWD: PRINTING_CMDS

 

Examples from Man Pages
 root            ALL = (ALL) ALL
 %wheel          ALL = (ALL) ALL
We let root and any user in group wheel run any command on any host as any user.
FULLTIMERS      ALL = NOPASSWD: ALL
Full time sysadmins may run any command on any host without authenticating.
WEBMASTERS  www = (www) ALL, (root) /usr/bin/su www
On the host www, any user in the WEBMASTERS User_Alias and may run any command
as user www (which owns the web pages) or simply su to www.
Important SUDO Commands

sudo -k

This command will remove the cached credential for the user and ask for the password in the next run sudo command.

sudo -l

Lists the current user permitted commands

sudo -Ul <user>

Lists the specified user permitted commands

sudo -v

Validates the user and increases the default cache for another default 15 min if that is set in configuration file.

sudo -V

Lists sudo version details and features

sudo -e

To edit the sudoers file , note export the preferred editor variable in bash before running this command.

export EDITOR=/bin/nano ;sudo visudo

In short sudo is a great feature in Nix operating system and it is a must known for system administrators.

Copyright © 2021 | SaitCare Hub SDN BHD