Thursday, November 14, 2019

IPC Command Examples in linux ?

IPC Inter-Process-Communication

Shared memory is the memory that may be accessed by multiple processes; i.e. a memory
region that can be shared between different processes and a better way of passing data
between two processes. Shared memory is the fastest form of Inter-Process
Communication which is currently available. Assume that the program will create a memory
portion, which another process can access (if permitted). A shared segment can be
attached multiple times by the same process. Whenever the memory is mapped into the
address space of the process, i.e. sharing the common memory region, the kernel will not
involve while passing data between the processes.


Different Types of IPCS

There are various IPC’s which allows a process to communicate with another processes,
either in the same computer or different computer in the same network.

Pipes – Provides a way for processes to communicate with each another by exchanging messages. Named pipes provide a way for processes running on different computer systems to communicate over the network.

Shared Memory – Processes can exchange values in the shared memory. One process will create a portion of memory which other process can access.

Message Queue – It is a structured and ordered list of memory segments where processes store or retrieve data.

Semaphores – Provides a synchronizing mechanism for processes that are accessing the same resource. No data is passed with a semaphore; it simply coordinates access to shared
resources.

List all the IPC facility

ipcs command with -a option lists all the IPC facilities which has read access for the current
process. It provides details about message queue, semaphore and shared memory.
ipcs -a


>ipcs -a

------ Shared Memory Segments --------
key        shmid owner      perms bytes nattch     status      
0x00000000 1671168    ravi 600 524288     2 dest         
0x00000000 393217     ravi 600 524288     2 dest         
0x00000000 1277954    ravi 600 524288     2 dest         
0x00000000 1310723    ravi 600 134217728  2 dest         
0x00000000 622596     ravi 600 524288     2 dest         
0x00000000 884741     ravi 600 524288     2 dest         
0x00000000 983046     ravi 600 393216     2 dest         
0x00000000 1081351    ravi 600 524288     2 dest         
0x00000000 2818056    ravi 600 524288     2 dest         

------ Semaphore Arrays --------
key        semid owner      perms nsems     

------ Message Queues --------
key        msqid owner      perms used-bytes   messages    


All the IPC facility has unique key and identifier, which is used to identify an IPC facility.

List all the Message Queue

ipcs with option -q, lists only message queues for which the current process has read
access.

ipcs -q

------ Message Queues --------
key        msqid owner      perms used-bytes   messages
0x000005a4 32768      root 644 0            0

List all the Semaphores

ipcs -s option is used to list the accessible semaphores.

ipcs -s

------ Semaphore Arrays --------
key        semid owner      perms nsems
0x0103eefd 0          root 664 1
0x0103eefe 32769      root 664 1

List all the Shared Memory

ipcs -m option with ipcs command lists the shared memories.

ipcs -m

------ Shared Memory Segments --------
key        shmid owner      perms bytes nattch     status      
0x00000000 1671168    ravi 600 524288     2 dest         
0x00000000 393217     ravi 600 524288     2 dest         
0x00000000 1277954    ravi 600 524288     2 dest         
0x00000000 1310723    ravi 600 134217728  2 dest     

Lists the Limits for IPC facility

ipcs -l option gives the system limits for each ipc facility.

ipcs -l

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 18014398509465599
max total shared memory (kbytes) = 18446744073642442748
min seg size (bytes) = 1

------ Semaphore Limits --------
max number of arrays = 32000
max semaphores per array = 32000
max semaphores system wide = 1024000000
max ops per semop call = 500
semaphore max value = 32767

------ Messages Limits --------
max queues system wide = 32000
max size of message (bytes) = 8192
default max size of queue (bytes) = 16384

List Creator and Owner Details for IPC Facility

ipcs -c option lists creator userid and groupid and owner userid and group id. This option
can be combined with -m, -s and -q to view the creator details for specific IPC facility.

ipcs -m -c

------ Shared Memory Segment Creators/Owners --------
shmid      perms cuid       cgid uid gid       
1671168    600 ravi        ravi ravi ravi       
393217     600 ravi        ravi ravi ravi       
1277954    600 ravi        ravi ravi ravi       
1310723    600 ravi        ravi ravi ravi       

Process ids that accessed IPC facility recently

ipcs -p option displays creator id, and process id which accessed the corresponding ipc
facility very recently.

ipcs -m -p

------ Shared Memory Creator/Last-op PIDs --------
shmid      owner cpid       lpid      
1671168    ravi 3124       10093     
393217     ravi 2647       2981      
1277954    ravi 2836       24419     
1310723    ravi 2836       24419     


Last Accessed Time

ipcs -t option displays last operation time in each ipc facility. This option can also be
combined with -m, -s or -q to print for specific type of ipc facility. For message queue, -t
option displays last sent and receive time, for shared memory it displays last attached
(portion of memory) and detached timestamp and for semaphore it displays last operation
and changed time details.

ipcs  -t

------ Shared Memory Attach/Detach/Change Times --------
shmid      owner attached             detached changed             
1671168    ravi Nov 14 08:12:16      Nov 14 08:12:16 Nov 6 19:20:58     
393217     ravi Nov  6 19:20:15 Nov  6 19:20:15 Nov 6 19:19:50     


 Status of current usage

ipcs with -u command displays current usage for all the IPC facility. This option can be
combined with a specific option to display the status for a particular IPC facility.

ipcs -u

------ Shared Memory Status --------
segments allocated 24
pages allocated 53664
pages resident  10535
pages swapped   691
Swap performance: 0 attempts 0 successes

------ Semaphore Status --------
used arrays = 49
allocated semaphores = 252

------ Messages: Status --------
allocated queues = 1
used headers = 0
used space = 0 bytes


How to Delete/remove a share memory ?

Delete/remove a share memory
ipcrm
usage: ipcrm [ [-q msqid] [-m shmid] [-s semid]
          [-Q msgkey] [-M shmkey] [-S semkey] ... ]

Delete/remove a Shared Memory segment
$ ipcrm -m 10715155

Delete a message queue
$ipcrm -s 163843 

Wednesday, October 9, 2019

How to Convert Epoch Seconds To the Current Time in linux?

Linux / UNIX: Convert Epoch Seconds To the Current Time

how you can obtain the UNIX epoch time (number of seconds since 1970-01-01
00:00:00 UTC) using the Linux bash "date" command. It also shows how you can
convert a UNIX epoch time to a human readable time.

Obtain UNIX epoch time using bash

Obtaining the UNIX epoch time using bash is easy. Use the build-in date command and
instruct it to output the number of seconds since 1970-01-01 00:00:00 UTC. You can do this
by passing a format string as parameter to the date command. The format string for UNIX
epoch time is '%s'.


Print Current UNIX Time
Type the following command to display the seconds since the epoch:

Syntax: date +%s

date +%s
1569569962


Convert Epoch To Current Time
Type the command:

date -d @Epoch

date -d @1569569962
date -d "1970-01-01 1569569962 sec GMT"

Sample outputs:

Fri Sep 27 13:09:22 IST 2019

To convert number of seconds back to a more readable form, use a command like this:

date -d @1569569962 +"%d-%m-%Y %T %z"


Using AWK



awk 'BEGIN { print strftime("%Y-%m-%d %H:%M:%S", 1569569962); }'

Output:

2019-09-27 13:09:22

Using Perl

perl -e 'print scalar(localtime(1569569962)), "\n"'

Output:

Fri Sep 27 13:09:22 2019

Tuesday, August 13, 2019

How to disable or enable hyper threading on linux?

Hyper threading uses processor resources more efficiently, enabling multiple threads to run
on each core. As a performance feature, it also increases processor throughput, improving
overall performance on threaded software. A single physical CPU core with hyper-threading
appears as two logical CPUs to an operating system.

The recommended way to disable HT is by disabling in the BIOS, if possible but this can
also be done via operating system using the below steps.


Disable HT on runtime for individual logical CPUs
Before starting let's check the lscpu stat

# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                32
On-line CPU(s) list:   0-31
Thread(s) per core:    2
Core(s) per socket:    8
Socket(s):             2


Here it shows that there are 2 threads per core so we know most likely hyper threading is
enabled


The following files will show all of the logical CPU's and their HT pair relationships
# grep -H . /sys/devices/system/cpu/cpu*/topology/thread_siblings_list

To determine which CPUs should be disabled, the threads running on the same CPU core
have to be identified. The files /sys/devices/system/cpu/cpuN/topology/thread_siblings_list
where N is the CPU socket number. This file will contain the logical (HT) CPU numbers for
each physical socket.

# grep -H . /sys/devices/system/cpu/cpu*/topology/thread_siblings_list | sort -n -t ',' -k 2 -u
/sys/devices/system/cpu/cpu0/topology/thread_siblings_list:0,16
/sys/devices/system/cpu/cpu17/topology/thread_siblings_list:1,17
/sys/devices/system/cpu/cpu18/topology/thread_siblings_list:2,18
/sys/devices/system/cpu/cpu19/topology/thread_siblings_list:3,19
/sys/devices/system/cpu/cpu20/topology/thread_siblings_list:4,20
/sys/devices/system/cpu/cpu21/topology/thread_siblings_list:5,21
/sys/devices/system/cpu/cpu22/topology/thread_siblings_list:6,22
/sys/devices/system/cpu/cpu23/topology/thread_siblings_list:7,23
/sys/devices/system/cpu/cpu24/topology/thread_siblings_list:8,24
/sys/devices/system/cpu/cpu25/topology/thread_siblings_list:9,25
/sys/devices/system/cpu/cpu10/topology/thread_siblings_list:10,26
/sys/devices/system/cpu/cpu11/topology/thread_siblings_list:11,27
/sys/devices/system/cpu/cpu12/topology/thread_siblings_list:12,28
/sys/devices/system/cpu/cpu13/topology/thread_siblings_list:13,29
/sys/devices/system/cpu/cpu14/topology/thread_siblings_list:14,30
/sys/devices/system/cpu/cpu15/topology/thread_siblings_list:15,31

This means that CPU0 and CPU16 are threads on the same core. The same for 1 and 17
and so on. Individual, logical HT CPUs could be turned off as needed for a specific
application that is bound to a physical core.


Or the following script would disable all of them, from logical CPU 16 through 31

# cat /tmp/disable_ht.sh
#!/bin/bash
for i in {16..31}; do
   echo "Disabling logical HT core $i."
   echo 0 > /sys/devices/system/cpu/cpu${i}/online;
done

To disable individual logical CPU use the below command and replace <cpu_id> with the id from (16..31)
echo 0 > /sys/devices/system/cpu/<cpu_id>/online

To re-enable the HT

# cat /tmp/enable_ht.sh
for i in {16..31}; do
   echo "Enabling logical HT core $i."
   echo 1 > /sys/devices/system/cpu/cpu${i}/online;

done