OpenVPN 2.x auto setup on Linux -RHEL8

 OpenVPN open source OpenVPN CLI program

The open source project client program can also connect to the Access Server. The package is available in most distributions and is known simply as openvpn. It supports the option to connect to multiple OpenVPN servers simultaneously, and it comes with a service component that can automatically and silently start any auto-login profiles it finds in the /etc/openvpn folder, even before a user has logged in. This service component can be set to automatically start at boot time with the tools available in your Linux distribution if supported. On Ubuntu and Debian, when you install the openvpn package, it is automatically configured to start at boot time.

To install the OpenVPN client on Linux, it is possible in many cases to just use the version that is in the software repository for the Linux distribution itself. If you run into any connectivity problems when using outdated software, it may be due to a possible lack of support for higher TLS versions in older versions of OpenVPN. Follow the instructions found on the open source openvpn community wiki if you wish to install the OpenVPN client on your Linux system.

After installing, you will need a connection profile. This is a file generated by your OpenVPN Access Server installation for your specific user account. It contains the required certificates and connection settings. Go to the Client web interface of your Access Server (the main address, not the /admin portion). Log in with your user credentials. You will be shown a list of files available to you for download. Pick the user-locked profile or the auto-login profile, and you will be sent a client.ovpn file. Save this file to your Linux operating system somewhere. OpenVPN Access Server supports server-locked, user-locked, and auto-login profiles, but the OpenVPN command line client is only able to connect with user-locked or auto-login connection profiles.

We are assuming you are going to start the connection through either the command line as a root user, or via the service daemon. If you want unprivileged users to be able to make a connection, take a look at the community wiki for more information on how to implement that. Here we are going to focus on the simplest implementation; run the connection as root user directly, or via the service daemon.

Start a connection with an auto-login profile manually:

Start a connection with an auto-login profile manually:

openvpn --config client.ovpn

Start a connection with a user-locked profile manually:

openvpn --config client.ovpn --auth-user-pass

If you use Google Authenticator or another extra factor authentication, add the auth-retry parameter:

openvpn --config client.ovpn --auth-user-pass --auth-retry interact

To start an auto-login connection via the service daemon, place client.ovpn in /etc/openvpn/ and rename the file. It must end with .conf as file extension. Ensure the service daemon is enabled to run after a reboot, and then simply reboot the system. The auto-login type profile will be picked up automatically and the connection will start itself. You can verify this by checking the output of the ifconfig command; you should see a tun0 network adapter in the list.

[root@openvpn]# systemctl enable openvpn@pok_zone10015-onsite

Created symlink from /etc/systemd/system/multi-user.target.wants/openvpn@pok_zone10015-onsite.service to /usr/lib/systemd/system/openvpn@.service.

[root@openvpn]# systemctl stop openvpn@pok_zone10015-onsite

[root@ openvpn]# systemctl start openvpn@pok_zone10015-onsite

Enter Auth Username: sachin@yahoo.com

Enter Auth Password: ********************

[root@ openvpn]#

[root@openvpn]# systemctl enable openvpn@pok_zone10015-onsite

Created symlink from /etc/systemd/system/multi-user.target.wants/openvpn@pok_zone10015-onsite.service to /usr/lib/systemd/system/openvpn@.service.

[root@ibmgpu07]# systemctl stop openvpn@pok_zone10015-onsite

[root@ibmgpu07 openvpn]# shutdown -r now

login as: root

root@ibmgpu07.eng.platformlab.ibm.com's password:

Last login: Mon Sep  6 06:47:18 2021 from 9.195.45.219

[root@ibmgpu07 ~]# systemctl status openvpn@pok_zone10015-onsite

● openvpn@pok_zone10015-onsite.service - OpenVPN Robust And Highly Flexible Tunneling Application On pok_zone10015/onsite

   Loaded: loaded (/usr/lib/systemd/system/openvpn@.service; enabled; vendor preset: disabled)

   Active: active (running) since Mon 2021-09-06 07:55:40 EDT; 14min ago

 Main PID: 2398 (openvpn)

   Status: "Initialization Sequence Completed"

   CGroup: /system.slice/system-openvpn.slice/openvpn@pok_zone10015-onsite.service

           └─2398 /usr/sbin/openvpn --cd /etc/openvpn/ --config pok_zone10015-onsite.conf

Sep 06 07:55:42 ibmgpu07 openvpn[2398]: Mon Sep  6 07:55:42 2021 Outgoing Data Channel: Using 160 bit message hash 'SHA1' for HMAC authentication

Sep 06 07:55:42 ibmgpu07 openvpn[2398]: Mon Sep  6 07:55:42 2021 Incoming Data Channel: Using 160 bit message hash 'SHA1' for HMAC authentication

Sep 06 07:55:42 ibmgpu07 openvpn[2398]: Mon Sep  6 07:55:42 2021 ROUTE_GATEWAY 9.21.55.2/255.255.255.0 IFACE=eno1 HWADDR=40:f2:e9:03:2c:0a

Sep 06 07:55:42 ibmgpu07 openvpn[2398]: Mon Sep  6 07:55:42 2021 TUN/TAP device tun0 opened

Sep 06 07:55:42 ibmgpu07 openvpn[2398]: Mon Sep  6 07:55:42 2021 TUN/TAP TX queue length set to 100

Sep 06 07:55:42 ibmgpu07 openvpn[2398]: Mon Sep  6 07:55:42 2021 /sbin/ip link set dev tun0 up mtu 1500

Sep 06 07:55:42 ibmgpu07 openvpn[2398]: Mon Sep  6 07:55:42 2021 /sbin/ip addr add dev tun0 10.255.15.2/24 broadcast 10.255.15.255

Sep 06 07:55:42 ibmgpu07 openvpn[2398]: Mon Sep  6 07:55:42 2021 /sbin/ip route add 10.20.160.0/22 via 10.255.15.1

Sep 06 07:55:42 ibmgpu07 openvpn[2398]: Mon Sep  6 07:55:42 2021 WARNING: this configuration may cache passwords in memory -- use the auth-nocach...vent this

Sep 06 07:55:42 ibmgpu07 openvpn[2398]: Mon Sep  6 07:55:42 2021 Initialization Sequence Completed

Hint: Some lines were ellipsized, use -l to show in full.

[root@ibmgpu07 ~]# ps -ef | grep openvpn

root      2398     1  0 07:55 ?        00:00:00 /usr/sbin/openvpn --cd /etc/openvpn/ --config pok_zone10015-onsite.conf

root      3894  3698  0 08:10 pts/0    00:00:00 grep --color=auto openvpn

[root@ibmgpu07 ~]# ip a | grep tun

9: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 100

    inet 10.255.15.2/24 brd 10.255.15.255 scope global tun0

[root@ibmgpu07 ~]#

==========================

Reference:

https://openvpn.net/vpn-server-resources/connecting-to-access-server-with-linux/

https://forums.openvpn.net/viewtopic.php?t=27419

https://openvpn.net/blog/openvpn-3-linux-and-auth-user-pass/

Bash : Select the column from list_file , decorate with double quotes at both side and put comma at the end of each line , concatenate

#!/bin/bash

## declare an array variable

declare -a arr=("Bcast" "Allreduce" "Reduce" "Allgather" "Allgatherv" "Allgatherv_int" "Scatter" "Scatterv" "Scatterv_int" "Gather" "Gatherv" "Gatherv_int" "Barrier" "Alltoall" "Alltoallv" "Alltoallv_int" "Scan" "Reduce_scatter" )

## now loop through the above array

j=0

echo "[collectives-smpi]" >> collectives.ini

for i in "${arr[@]}"

do

   echo "$i"

   ALGO=${i^^}

   ALL=$(awk "/^$j;/ "'{ print $0 }' OR=' ' all-coll.txt | cut -d ';' -f 3 |  awk '{ print "\""$0"\""}' | sed '$!s/$/,/' | awk '//' ORS=' ')

   echo "ALL_$ALGO=$ALL" >> collectives.ini

   ((j=j+1))

   echo "value of j is $j"

done

#----------------------------------------------------------------------

NOTE: You can also use 

   ALL_$i=$(awk '/^$i/' OR=' ' ${_TEST_DIR}/all-coll.txt | cut -d ';' -f 3 | sed -e 's/^\|$/"/g' | sed '$!s/$/,/' | tr -d '\n')

#----------------------------------------------------------------------

[Test get: fvt_bigMPI]

module = Copytree

copytree_directory = &getenv("fvt_bigMPI_src")

#----------------------------------------------------------------------

[Test build: fvt_bigMPI]

test_get = fvt_bigMPI

save_stdout_on_success = 1

merge_stdout_stderr = 1

module = Shell

shell_build_command = <<EOT

make compile

make generate

cp BigMPI/test/all-coll.txt $GLOBAL_SCRATCH/

declare -a arr=("Bcast" "Allreduce" "Reduce" "Allgather" "Allgatherv" "Allgatherv_int" "Scatter" "Scatterv" "Scatterv_int" "Gather" "Gatherv" "Gatherv_int" "Barrier" "Alltoall" "Alltoallv" "Alltoallv_int" "Scan" "Reduce_scatter" )

j=0

for i in "${arr[@]}"

do

   ALGO=${i^^}

   ALL=$(awk "/^$j;/ "'{ print $0 }' OR=' ' $GLOBAL_SCRATCH/all-coll.txt | cut -d ';' -f 3 | sed -e 's/^\|$/"/g' | sed '$!s/$/,/' | awk '//' ORS=' ')

   echo "ALL_$ALGO=$ALL" >> $GLOBAL_SCRATCH/collectives/collectives.ini

   ((j=j+1))

done

EOT

===============

 cat $MTT_CONF/collectives/collectives.ini

[collectives-smpi]

ALL_BCAST="EXT:Broadcast:EXT:EXT", "I0:2-nary:P2P:P2P", "I0:2-nomial:P2P:P2P", "I0:3-nary:P2P:P2P", "I0:3-nomial:P2P:P2P", "I0:4-nary:P2P:P2P", "I0:4-nomial:P2P:P2P", "I0:HybridBinomial:SHMEM:P2P", "I0:HybridMultiColor:Shmem:P2P", "I0:MultiColor:P2P:P2P", "I0:SequenceBased_Binomial:P2P:P2P"

ALL_ALLREDUCE="EXT:Allreduce:EXT:EXT", "I0:Binomial:P2P:P2P", "I0:HybridBinomial4:SHMEM:P2P", "I0:HybridBinomial:SHMEM:P2P", "I0:HybridMultiColor:SHMEM:P2P", "I0:MultiColor:P2P:P2P", "I0:MultinomialRadix4:P2P:P2P", "I0:MultinomialRadix8:P2P:P2P", "I0:SequenceBased_Binomial4:P2P:P2P", "I0:SequenceBased_Binomial:P2P:P2P", "I0:SequenceBased_Tree4:P2P:P2P", "I0:SequenceBased_Tree:P2P:P2P"

ALL_REDUCE="EXT:Reduce:EXT:EXT", "I0:HybridAsyncBinomial4:Shmem:P2P", "I0:HybridAsyncBinomial:Shmem:P2P", "I0:HybridMultiColor:SHMEM:P2P", "I0:MultiColor:P2P:P2P", "I0:SequenceBased_Binomial4:P2P:P2P", "I0:SequenceBased_Binomial:P2P:P2P", "I0:SequenceBased_Tree4:P2P:P2P", "I0:SequenceBased_Tree:P2P:P2P"

ALL_ALLGATHER="EXT:Allgather:EXT:EXT", "I0:Binomial:P2P:P2P", "I0:Flat:P2P:P2P", "I0:HybridFlat:Shmem:P2P", "I0:Ring:P2P:P2P"

ALL_ALLGATHERV="EXT:Allgatherv:EXT:EXT", "I0:HybridFlat:Shmem:P2P", "I0:Ring:P2P:P2P"

ALL_ALLGATHERV_INT="EXT:Allgatherv_int:EXT:EXT", "I0:HybridFlat:Shmem:P2P", "I0:Ring:P2P:P2P"

ALL_SCATTER="EXT:Scatter:EXT:EXT", "I0:Flat:P2P:P2P", "I0:ScatterMulticolorBcast:P2P:P2P"

ALL_SCATTERV="EXT:Scatterv:EXT:EXT", "I0:ScattervMulticolorBcast:P2P:P2P", "I0:SoftwareTree:P2P:P2P"

ALL_SCATTERV_INT="EXT:Scatterv_int:EXT:EXT", "I0:ScattervintMulticolorBcast:P2P:P2P", "I0:SoftwareTree:P2P:P2P"

ALL_GATHER="EXT:Gather:EXT:EXT", "I0:AsyncFlat:P2P:P2P", "I0:Binomial:P2P:P2P", "I0:Flat:P2P:P2P", "I0:HybridFlat:SHMEM:P2P"

ALL_GATHERV="EXT:Gatherv:EXT:EXT", "I0:FlatTree:P2P:P2P", "I0:HybridFlat:SHMEM:P2P"

ALL_GATHERV_INT="EXT:Gatherv_int:EXT:EXT", "I0:FlatTree:P2P:P2P", "I0:HybridFlat:SHMEM:P2P"

ALL_BARRIER="EXT:Barrier:EXT:EXT", "I0:Binomial4:P2P:P2P", "I0:Binomial:P2P:P2P", "I0:HybridBinomial:SHMEM:P2P", "I0:MultiLeaderBarrier:P2P:P2P"

ALL_ALLTOALL="EXT:Alltoall:EXT:EXT", "I0:Bruck:P2P:P2P", "I0:M2MComposite:P2P:P2P"

ALL_ALLTOALLV="EXT:Alltoallv:EXT:EXT", "I0:M2MComposite:P2P:P2P"

ALL_ALLTOALLV_INT="EXT:Alltoallv_int:EXT:EXT", "I0:M2MComposite:P2P:P2P"

ALL_SCAN="EXT:Scan:EXT:EXT", "I0:Binomial:P2P:P2P"

ALL_REDUCE_SCATTER="EXT:Reduce_scatter:EXT:EXT", "I0:Hybrid_Sequence_Tree_On_Allreduce:P2P:P2P", "I0:Sequence_Tree:P2P:P2P", "I0:Sequence_Tree_On_Allreduce:P2P:P2P" 

Spectrum Scale: How to set systemd to auto start GPFS daemons after reboot

GPFS comes with a systemd unit file (/usr/lpp/mmfs/lib/systemd/gpfs.service) that comes with the RPMs by default.

systemctl status gpfs.service


[root@IBMPOWER_HOST1 ~]#  mmgetstate -a

 Node number  Node name        GPFS state
-------------------------------------------
       1      IBMPOWER_HOST1        active
       2      IBMPOWER_HOST2        active
       3      IBMPOWER_HOST3        active
[root@IBMPOWER_HOST1 ~]# systemctl status gpfs.service
● gpfs.service - General Parallel File System
   Loaded: loaded (/usr/lib/systemd/system/gpfs.service; disabled; vendor preset: disabled)
   Active: active (running) since Sun 2020-04-19 00:39:51 EDT; 5 days ago
  Process: 65927 ExecStart=/usr/lpp/mmfs/bin/mmremote startSubsys systemd $STARTSUBSYS_ARGS (code=exited, status=0/SUCCESS)
 Main PID: 65959 (runmmfs)
    Tasks: 689
   Memory: 1.0G
   CGroup: /system.slice/gpfs.service
           ├─65959 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/runmmfs
           └─66297 /usr/lpp/mmfs/bin/mmfsd

Apr 19 00:39:51 IBMPOWER_HOST1 systemd[1]: Started General Parallel File System.
Apr 19 00:39:55 IBMPOWER_HOST1 mmfs[66297]: [N] CCR: failed to connect to node 9.114.75.217:1191 (sock 53 err 79)
Apr 19 00:39:56 IBMPOWER_HOST1 mmfs[66297]: [N] Connecting to 9.114.75.217 IBMPOWER_HOST3 <c0p0>
Apr 19 00:39:56 IBMPOWER_HOST1 mmfs[66297]: [N] This node (9.114.75.216 (IBMPOWER_HOST1)) is now Cluster Manager for SMPICI_gpfs.IBMPOWER_HOST1.
Apr 19 00:39:56 IBMPOWER_HOST1 mmfs[66297]: [N] mmfsd ready
Apr 19 00:39:57 IBMPOWER_HOST1 mmfs[66297]: [N] Node 9.114.75.217 (IBMPOWER_HOST3) appointed as manager for gpfs_fs.
Apr 20 16:08:19 IBMPOWER_HOST1 mmfs[66297]: [N] Node 9.114.75.215 (IBMPOWER_HOST2) lease renewal is overdue. Pinging to check if it is alive
Apr 20 18:23:10 IBMPOWER_HOST1 mmfs[66297]: [N] Node 9.114.75.217 (IBMPOWER_HOST3) lease renewal is overdue. Pinging to check if it is alive
Apr 20 20:25:15 IBMPOWER_HOST1 mmfs[66297]: [N] Node 9.114.75.217 (IBMPOWER_HOST3) lease renewal is overdue. Pinging to check if it is alive
Apr 20 22:46:10 IBMPOWER_HOST1 mmfs[66297]: [N] Node 9.114.75.217 (IBMPOWER_HOST3) lease renewal is overdue. Pinging to check if it is alive
[root@IBMPOWER_HOST1 ~]#


[root@IBMPOWER_HOST1 ~]# systemctl is-active gpfs.service
active
[root@IBMPOWER_HOST1 ~]# systemctl is-enabled gpfs.service
disabled
[root@IBMPOWER_HOST1 ~]#
[root@IBMPOWER_HOST1 ~]# systemctl is-failed gpfs.service
active
[root@IBMPOWER_HOST1 ~]#



This will return active if it is running properly or failed if an error occurred.
If the unit was intentionally stopped, it may return unknown or inactive


---------------------------------

[root@IBMPOWER_HOST1 ~]# rm -rf /etc/systemd/system/multi-user.target.wants/gpfs.service
[root@IBMPOWER_HOST1 ~]# systemctl enable  gpfs.service
Created symlink from /etc/systemd/system/multi-user.target.wants/gpfs.service to /usr/lib/systemd/system/gpfs.service.
[root@IBMPOWER_HOST1 ~]# ls -alsrt /etc/systemd/system/multi-user.target.wants/gpfs.service
0 lrwxrwxrwx 1 root root 36 Apr 24 03:41 /etc/systemd/system/multi-user.target.wants/gpfs.service -> /usr/lib/systemd/system/gpfs.service
[root@IBMPOWER_HOST1 ~]#

This will create a symbolic link from the system’s copy of the service file (usually in /lib/systemd/system or /etc/systemd/system)
into the location on disk where systemd looks for autostart files (usually /etc/systemd/system/some_target.target.wants.


--------------------------------------------
[root@IBMPOWER_HOST1 ~]# systemctl enable  gpfs.service
Created symlink from /etc/systemd/system/multi-user.target.wants/gpfs.service to /usr/lib/systemd/system/gpfs.service.
[root@IBMPOWER_HOST1 ~]#

[root@IBMPOWER_HOST1 ~]# systemctl enable  gpfs.service
Created symlink from /etc/systemd/system/multi-user.target.wants/gpfs.service to /usr/lib/systemd/system/gpfs.service.
[root@IBMPOWER_HOST1 ~]# ls -alsrt /etc/systemd/system/multi-user.target.wants/gpfs.service
0 lrwxrwxrwx 1 root root 36 Apr 24 03:41 /etc/systemd/system/multi-user.target.wants/gpfs.service -> /usr/lib/systemd/system/gpfs.service
[root@IBMPOWER_HOST1 ~]#


Lets check the status

[root@IBMPOWER_HOST1 ~]# systemctl status gpfs.service
● gpfs.service - General Parallel File System
   Loaded: loaded (/usr/lib/systemd/system/gpfs.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2020-04-24 03:35:55 EDT; 10min ago
 Main PID: 59025 (runmmfs)
   CGroup: /system.slice/gpfs.service
           ├─59025 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/runmmfs
           └─59548 /usr/lpp/mmfs/bin/mmfsd

Apr 24 03:35:55 IBMPOWER_HOST1 systemd[1]: Starting General Parallel File System...
Apr 24 03:35:55 IBMPOWER_HOST1 systemd[1]: Can't open PID file /var/mmfs/gen/runmmfsPid (yet?) after start: No such file or directory
Apr 24 03:35:55 IBMPOWER_HOST1 systemd[1]: Started General Parallel File System.
Apr 24 03:36:00 IBMPOWER_HOST1 mmfs[59548]: [N] Connecting to 9.114.75.215 IBMPOWER_HOST2 <c0p2>
Apr 24 03:36:00 IBMPOWER_HOST1 mmfs[59548]: [N] mmfsd ready
Apr 24 03:36:01 IBMPOWER_HOST1 mmfs[59548]: [N] Connecting to 9.114.75.217 IBMPOWER_HOST3 <c0n0>
[root@IBMPOWER_HOST1 ~]#




[root@IBMPOWER_HOST1 ~]#
[root@IBMPOWER_HOST1 ~]#  mmgetstate -a

 Node number  Node name        GPFS state
-------------------------------------------
       1      IBMPOWER_HOST1        active
       2      IBMPOWER_HOST2        active
       3      IBMPOWER_HOST3        active
[root@IBMPOWER_HOST1 ~]#
[root@IBMPOWER_HOST1 ~]# systemctl is-enabled gpfs.service
enabled
[root@IBMPOWER_HOST1 ~]#


How to auto mount the filesystem when GPFS daemon starts - if you have not done at installation 


export PATH=$PATH:/usr/lpp/mmfs/bin

[root@IBMPOWER_HOST1 ~]#  mmgetstate -a

 Node number  Node name        GPFS state
-------------------------------------------
       1      IBMPOWER_HOST1        active
       2      IBMPOWER_HOST2        active
       3      IBMPOWER_HOST3        active
[root@IBMPOWER_HOST1 ~]# mmchfs gpfs_fs -A yes
mmchfs: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.
[root@IBMPOWER_HOST1 ~]#

-A {yes | no | automount}
    Indicates when the file system is to be mounted:

    yes
        When the GPFS daemon starts.


Verify:

[root@IBMPOWER_HOST1 ~]# mmlsfs gpfs_fs  -A
flag                value                    description
------------------- ------------------------ -----------------------------------
 -A                 yes                      Automatic mount option
[root@IBMPOWER_HOST1 ~]#

How to Auto-load and auto-mount Spectrum scale in simple steps ?

[root@IBMPOWER_HOST1 ~]# mmchconfig autoload=yes
mmchconfig: Command successfully completed
mmchconfig: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.
[root@IBMPOWER_HOST1 ~]#

 Reference:
https://www.ibm.com/support/knowledgecenter/STXKQY_5.0.4/com.ibm.spectrum.scale.v5r04.doc/bl1adm_mmchfs.htm
https://www.ibm.com/support/knowledgecenter/en/SSCKLT_2.0.0/UG/sec_ug_starting_scale_in_system_file.html

Bash Script: How to check the status of background processes

Example 1 :  Failure case with exit 24

[smpici@c712f6n06 myscripts]$ cat simple_bkGroundproc.sh
#!/bin/sh

cmd() { sleep 5; exit 24; }

cmd &   # Run the long running process
pid=$!  # Record the pid

# Spawn a process that coninually reports that the command is still running
while echo "$(date): $pid is still running"; do sleep 1; done &
echoer=$!

# Set a trap to kill the reporter when the process finishes
trap 'kill $echoer' 0

# Wait for the process to finish
if wait $pid; then
    echo "cmd succeeded"
else
    echo "cmd FAILED!! (returned $?)"
fi
[sachin@host1 myscripts]$ ./simple_bkGroundproc.sh
Wed Mar  4 06:36:34 EST 2020: 96853 is still running
Wed Mar  4 06:36:35 EST 2020: 96853 is still running
Wed Mar  4 06:36:36 EST 2020: 96853 is still running
Wed Mar  4 06:36:37 EST 2020: 96853 is still running
Wed Mar  4 06:36:38 EST 2020: 96853 is still running
cmd FAILED!! (returned 24)
[sachin@host1 myscripts]$

--------------------------------------------

Example 1 :  Passed case with exit 0


#!/bin/sh

cmd() { sleep 5; exit 0; }

cmd &   # Run the long running process
pid=$!  # Record the pid

# Spawn a process that coninually reports that the command is still running
while echo "$(date): $pid is still running"; do sleep 1; done &
echoer=$!

# Set a trap to kill the reporter when the process finishes
trap 'kill $echoer' 0

# Wait for the process to finish
if wait $pid; then
    echo "cmd succeeded"
else
    echo "cmd FAILED!! (returned $?)"
fi

------------
OUTPUT
[sachin@host1 myscripts]$ ./simple_bkGroundproc.sh
Wed Mar  4 06:37:04 EST 2020: 97118 is still running
Wed Mar  4 06:37:05 EST 2020: 97118 is still running
Wed Mar  4 06:37:06 EST 2020: 97118 is still running
Wed Mar  4 06:37:07 EST 2020: 97118 is still running
Wed Mar  4 06:37:08 EST 2020: 97118 is still running
cmd succeeded


 ++++++++++++++++++++++++++++++++++++++++

Example 3 :

#! /bin/bash
fvt_test_arg="Default HCOLL IBV_ENABLE pami_noib"
IFS=', ' read -r -a fvt_test_arg_array <<< "$fvt_test_arg"

items="1 2 3 4"
pids=""

for item in $items; do
    sleep $item &
    pids+="$! "
done
echo "list of pids $pids"
i=0


for pid in $pids; do
    wait $pid
    if [ $? -eq 0 ]; then
        echo "${fvt_test_arg_array[i]} SUCCESS - Job $pid exited with a status of $?"
    else
        echo "${fvt_test_arg_array[i]} FAILED - Job $pid exited with a status of $?"
    fi
    i=$((i+1))
done
-----------------------
OUTPUT:
[sachin@host1 myscripts]$ ./pid_array2.sh
list of pids 81822 81823 81824 81825
Default SUCCESS - Job 81822 exited with a status of 0
HCOLL SUCCESS - Job 81823 exited with a status of 0
IBV_ENABLE SUCCESS - Job 81824 exited with a status of 0
pami_noib SUCCESS - Job 81825 exited with a status of 0
[sachin@host1 myscripts]$

-----------------------

Reference:
https://stackoverflow.com/questions/1570262/get-exit-code-of-a-background-process

Bash script : How to split string to array and count/print array elements based on index

IFS    The Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is <space><tab><newline>.


Example 1 :

#!/bin/bash
fvt_test_arg="DEFAULT HCOLL IBV PAMI_NOIB"
echo "This is fvt_test_arg : $fvt_test_arg"
IFS=', ' read -r -a fvt_test_arg_array <<< "$fvt_test_arg"
echo "This is fvt_test_arg_array : ${fvt_test_arg_array[@]}"
echo "Number of elements in the array : ${#fvt_test_arg_array[@]}"
echo "1st element is :${fvt_test_arg_array[0]}"
echo "2nd element is :${fvt_test_arg_array[1]}"
echo "3rd element is :${fvt_test_arg_array[2]}"
echo "4th element is :${fvt_test_arg_array[3]}"


----------------------------
OUTPUT:

[sachin@host1 myscripts]$ ./list4.sh
This is fvt_test_arg : DEFAULT HCOLL IBV PAMI_NOIB        ===> String
This is fvt_test_arg_array : DEFAULT HCOLL IBV PAMI_NOIB  ====> array
Number of elements in the array : 4
1st element is :DEFAULT
2nd element is :HCOLL
3rd element is :IBV
4th element is :PAMI_NOIB
[sachin@host1 myscripts]$

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
EXAMPLE 2:

#!/bin/bash
fvt_test_arg="DEFAULT HCOLL IBV PAMI_NOIB"
echo "This is fvt_test_arg : $fvt_test_arg"
IFS=', ' read -r -a fvt_test_arg_array <<< "$fvt_test_arg"
echo "This is fvt_test_arg_array : ${fvt_test_arg_array[@]}"
echo "Number of elements in the array : ${#fvt_test_arg_array[@]}"
echo "1st element is :${fvt_test_arg_array[0]}"
echo "2nd element is :${fvt_test_arg_array[1]}"
echo "3rd element is :${fvt_test_arg_array[2]}"
echo "4th element is :${fvt_test_arg_array[3]}"

pids="1111 2222 3333 4444"
i=0
for pid in ${pids[@]}; do
    echo " $pid is the PID of the process for :${fvt_test_arg_array[i]} is successful"
    i=$((i+1))
done

 ---------------
OUTPUT:
[sachin@host1 myscripts]$ ./list5.sh
This is fvt_test_arg : DEFAULT HCOLL IBV PAMI_NOIB
This is fvt_test_arg_array : DEFAULT HCOLL IBV PAMI_NOIB
Number of elements in the array : 4
1st element is :DEFAULT
2nd element is :HCOLL
3rd element is :IBV
4th element is :PAMI_NOIB
 1111 is the PID of the process for :DEFAULT is successful
 2222 is the PID of the process for :HCOLL is successful
 3333 is the PID of the process for :IBV is successful
 4444 is the PID of the process for :PAMI_NOIB is successful
[smpici@c712f6n06 myscripts]$


------------------------------------
 EXAMPLE  3:

#!/bin/bash

declare -a num
declare -a words

num=(ddd1 dd2 dd3 dd4 dd5 dd6 dd7)
words=(one two three four five six seven)

n=${#num[@]}
echo $n
for ((i=0;i<$n;i++)); do
  echo ":${num[$i]} :${words[$i]}:"
done
[sachin@host1 myscripts]$

--------------------------

OUTPUT:
[sachin@host1 myscripts]$ ./list3.sh
7
:ddd1 :one:
:dd2 :two:
:dd3 :three:
:dd4 :four:
:dd5 :five:
:dd6 :six:
:dd7 :seven:
-----------------------------------------------

Reference:
https://stackoverflow.com/questions/10586153/split-string-into-an-array-in-bash/13196466

VNC Server Setup

Procedure to setup VNC server to access the GUI ( x Server)  from VNC client remotely.
===============================================================
1) cat /etc/sysconfig/vncservers

 VNCSERVERS="1:sachin"  ==>  PORT 2 on secured env will throw connection error
 VNCSERVERARGS[2]="-geometry 800x600 -nolisten tcp -localhost"




2) [root]# su - sachin
-bash-4.1$ whoami
sachin
-bash-4.1$ vncpasswd
Password:
Verify:
-bash-4.1$



3) service vncserver stop

4) service vncserver start

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++



 [root@ ~]# vncserver -help

usage: vncserver [:<number>] [-name <desktop-name>] [-depth <depth>]
                 [-geometry <width>x<height>]
                 [-pixelformat rgbNNN|bgrNNN]
                 [-fp <font-path>]
                 [-fg]
                 [-autokill]
                 <Xvnc-options>...

       vncserver -kill <X-display>

       vncserver -list

[root@ ~]#


=====================================================


[root@~]# vncserver -list

TigerVNC server sessions:

X DISPLAY #     PROCESS ID
[root@ ~]#



=============================================================


[root@]# service vncserver status
Xvnc is stopped
[root@]# service vncserver start
Starting VNC server: no displays configured                [FAILED]
[root@ ]#



============================================
[root@ sachin]# cat /etc/sysconfig/vncservers
# The VNCSERVERS variable is a list of display:user pairs.
#
# Uncomment the lines below to start a VNC server on display :2
# as my 'myusername' (adjust this to your own).  You will also
# need to set a VNC password; run 'man vncpasswd' to see how
# to do that.
#
# DO NOT RUN THIS SERVICE if your local area network is
# untrusted!  For a secure way of using VNC, see this URL:
# http://kbase.redhat.com/faq/docs/DOC-7028

# Use "-nolisten tcp" to prevent X connections to your VNC server via TCP.

# Use "-localhost" to prevent remote VNC clients connecting except when
# doing so through a secure tunnel.  See the "-via" option in the
# `man vncviewer' manual page.

# VNCSERVERS="2:myusername"
# VNCSERVERARGS[2]="-geometry 800x600 -nolisten tcp -localhost"


 VNCSERVERS="1:sachin"
 VNCSERVERARGS[2]="-geometry 800x600 -nolisten tcp -localhost"


[root@]#

NOTE: PORT 2 on secured env will throw connection error
============================================
[root@sachin]# service vncserver status
Xvnc is stopped
[root@sachin]# service vncserver start
Starting VNC server: 2:sachin xauth:  creating new authority file /myhome/sachin/.Xauthority

New 'bgqxxx:2 (sachin)' desktop is bgqxxxa:2

Creating default startup script /bglhome/sachinpb/.vnc/xstartup
Starting applications specified in /bglhome/sachin/.vnc/xstartup
Log file is /bglhome/sachin/.vnc/bgqxxx:2.log

                                                           [  OK  ]
[root@ sachin]#

=====================================================
If you are not able to connect  VNC server  from VNC client , then Change the PORT to 1  in config file and start vncserver again.


[root@sachin]# ps -ef | grep vnc
sachin  2630     1  0 23:17 ?        00:00:01 /usr/bin/Xvnc :2 -desktop bgqxxx:2 (sachin) -auth /myhome/sachin/.Xauthority -geometry 800x600 -rfbwait 30000 -rfbauth /myhome/sachin/.vnc/passwd -rfbport 5902 -fp catalogue:/etc/X11/fontpath.d -pn -nolisten tcp -localhost
sachin  2649  2647  0 23:17 ?        00:00:00 vncconfig -iconic
root      3680 10443  0 23:24 pts/8    00:00:00 grep --color=auto vnc
[root@ sachin]#
=======================================================================

Example : showing TotalView  screen  for MPI_hello_world_Program

NOTE: Putty would throw this error and needs VNC client  to connect X server   to use  TotalView Graphical User Interface .

-bash-4.1$ export LM_LICENSE_FILE=/myusr/mtn/totalview/toolworks/flexlm-10.8.0-3/license.dat
-bash-4.1$ /myusr/mtn/totalview/current/bin/totalview -my_io_interface 172.16.1.2 runjob -a --exe /myusr/mtn/a.out --block R-M-N
Unable to open X display. Please check your $DISPLAY environment
variable to ensure that it is defined correctly and that you are
authorized to connect to this X server.
-bash-4.1$
++++++++++++++++++++++++++END ++++++++++++++++++++++++

Reference :
http://linuxconfig.org/configuring-vnc-server-access-on-a-redhat-linux