Wireguard on Raspberry

Written by pmd - - no comments

I have tried to use Wireguard following two guides :

  1. From this forum thread, without succes: Guide: Install Wireguard On Raspberry latest releases
  2. From this blog article, without succes as well: Installing and Configuring WireGuard on Raspberry Pi OS (August 2020)

This can be used as well to generate wireguard peers configurations + QR codes: Wireguard Tools

=> no successfull handshake between server (raspberry) and peers (Android and Windows 10).

Configuration

Server:

$ sudo cat /etc/wireguard/wg0.conf
[Interface]
Address = 192.168.99.1/24
ListenPort = 58280
PrivateKey = gNVxJe7Se842IiOR5GsXeM4sHcacGhPATIdQCgqP8Wa=
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
[Peer]
PublicKey = OQmmvh9/8PDWFIpOEzVWzOZ1HXQ48+10vONFlUNb0ia=
AllowedIPs = 192.168.99.2/32
[Peer]
PublicKey = N9VPXnH8hip4sJGGWm4ziLFWD5ZAveoj7H5oH8OgsHa=
AllowedIPs = 192.168.99.3/32

Peer 1:

$ cat ~/wg_config/users/client1/client.conf
[Interface]
Address = 192.168.99.2/24
PrivateKey = 6OfJPX1ZQCFu08fTy2uU6JdgUf/qXgzBoTtX/tCYX3a=

[Peer]
PublicKey = b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
AllowedIPs = 192.168.99.1/32, 192.168.1.0/24
Endpoint = adress.ddns.net:58280

Peer 2:

$ cat ~/wg_config/users/client2/client.conf
[Interface]
Address = 192.168.99.3/24
PrivateKey = uB+g5H0kbyI07kHdAajcQUE8VqMTaNqqiu0yj6BrH1a=

[Peer]
PublicKey = b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
AllowedIPs = 192.168.99.1/32, 192.168.1.0/24
Endpoint = adress.ddns.net:58280

 

Troubleshooting

12/10/2020

UDP correctly forwarded

I verified UDP port was correctly forwarded by my ISP modem/router, following Test whether UDP port is open: simple UDP server and client

Server side:

$ nc -l -u -p 58280

Client side:

$ nc -u servname_or_ip 58280

Checking if packets arrive to server

Listening on specific interface and on precise port of the server:

$ sudo tcpdump -i eth0 'port 58280'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:56:36.474701 IP 92.88.90.88.56188 > 192.168.1.201.58280: UDP, length 148
20:56:36.476725 IP 192.168.1.201.58280 > 92.88.90.88.56188: UDP, length 92
20:57:34.066017 IP 92.88.90.88.51673 > 192.168.1.201.58280: UDP, length 148
20:57:34.070037 IP 192.168.1.201.58280 > 92.88.90.88.51673: UDP, length 92

Here I tried two times to connect a peer to the server while pinging Wireguard server IP (192.168.99.1) from peer.

17/10/2020

Recording packets using tcpdump on both client and server sides

CLIENT: in a country potentially blocking VPN stuff
SERVER: in France, probably not blocking anything

I have generated another peer configuration. This time it is not a windows, not an android, but an openwrt router using same .
I have fixed the port in use for the wireguard client on openwrt in order to listen WAN interface on 51820.

What is observed on CLIENT openwrt side:

root@OpenWrt:~# tcpdump -i eth1 'port 51820'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
14:37:45.906247 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:37:46.025023 IP raspberry.abo.wanadoo.fr.58280 > 192.168.1.102.51820: UDP, length 92
14:37:46.038821 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:38:11.087567 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:38:36.687153 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:39:02.286884 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:39:27.887315 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:39:53.487145 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32
14:39:53.498819 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:39:59.257666 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:04.377588 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:10.138437 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:15.257703 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:21.017550 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:26.782109 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:31.897640 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:37.659644 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:42.777571 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:48.537585 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:40:54.298502 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:00.057651 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:05.177582 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:10.937544 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:16.697736 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:22.457569 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:28.220105 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:33.977597 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:41:39.097547 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
14:42:04.697538 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 148
^C
29 packets captured
29 packets received by filter
0 packets dropped by kernel
root@OpenWrt:~# wg
interface: WG0
public key: OQmmvh9/8PDWFIpOEzVWzOZ1HXQ48+10vONFlUNb0ia=
private key: (hidden)
listening port: 51820

peer: b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
endpoint: raspberry.abo.wanadoo.fr:58280
allowed ips: 192.168.99.1/32
latest handshake: 32 minutes, 12 seconds ago
transfer: 92 B received, 40.80 KiB sent
persistent keepalive: every 25 seconds
root@OpenWrt:~#
  • Only first two captured packets were seen by SERVER side.
  • These two captured packets are enough to declare successful handshake on CLIENT side.

What is observed on SERVER raspberry pi side:

pi@raspberrypi:~ $ sudo tcpdump -i eth0 'port 58280'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
16:37:45.924082 IP 93.88.83.27.51820 > 192.168.1.201.58280: UDP, length 148
16:37:45.928019 IP 192.168.1.201.58280 > 93.88.83.27.51820: UDP, length 92
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel
pi@raspberrypi:~ $ sudo wg
interface: wg0
public key: b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
private key: (hidden)
listening port: 58280

peer: OQmmvh9/8PDWFIpOEzVWzOZ1HXQ48+10vONFlUNb0ia=
endpoint: 93.88.83.27:51820
allowed ips: 192.168.99.2/32
transfer: 888 B received, 552 B sent
pi@raspberrypi:~ $
  • Only first two packets captured by CLIENT are seen as well on SERVER side.
  • Handshake is not declared successful on SERVER side.
  • Why SERVER is not seeing following packets ??? If I restart CLIENT, SERVER does not see packets for new handshake unless port used by CLIENT changes.
  • Why SERVER is not seeing anything from this CLIENT packet: " 14:37:46.038821 IP 192.168.1.102.51820 > raspberry.abo.wanadoo.fr.58280: UDP, length 32 "

Trying to connect from local country

CLIENT: in a country potentially blocking VPN stuff Computer using Windows in France, probably not blocking anything
SERVER: in France, probably not blocking anything

pi@raspberrypi:~ $ ip route show table 42
default via 192.168.1.1 dev eth0 proto dhcp src 192.168.1.201 metric 202
192.168.1.0/24 dev eth0 proto dhcp scope link src 192.168.1.201 metric 202
192.168.99.0/24 dev wg0 proto kernel scope link src 192.168.99.1
pi@raspberrypi:~ $ sudo tcpdump -i eth0 'port 58280'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
19:58:01.734652 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 148  # handshake
19:58:01.741670 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 92   # handshake
19:58:01.781909 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 96   # ping from client
19:58:01.782398 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 96   # server answers
19:58:02.893737 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 96   # ping from client
19:58:02.894315 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 96   # server answers
19:58:03.822017 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 96   # ping from client
19:58:03.822643 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 96   # server answers
19:58:05.793794 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 96   # ping from client
19:58:05.794394 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 96   # server answers
19:58:15.839250 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 32   # ???
19:58:51.032841 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 128  # server pings client
19:58:51.123963 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 128  # client answers
19:58:52.033771 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 128  # server pings client
19:58:52.090988 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 128  # client answers
19:58:53.035792 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 128  # server pings client
19:58:53.135887 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 128  # client answers
19:58:54.037607 IP 192.168.1.201.58280 > device.mobile.abo.orange.fr.51706: UDP, length 128  # server pings client
19:58:54.076616 IP device.mobile.abo.orange.fr.51706 > 192.168.1.201.58280: UDP, length 128  # client answers
[...]
[...]
[...]
pi@raspberrypi:~ $ sudo wg
interface: wg0
  public key: b6kqDH4pjAdK0LqPrEF4Fc9d4XxR0Eb3kSk9rzdEKma=
  private key: (hidden)
  listening port: 58280

peer: xxMWH9tZDwCNXPbErQxBuDejgkxNU1QOm9vFezUBeSa=
  endpoint: device.mobile.abo.orange.fr :51706
  allowed ips: 192.168.99.6/32
  latest handshake: 19 seconds ago
  transfer: 564 B received, 476 B sent

=> It worked ! I conclude that wireguard is blocked in the country where the client is.

Interesting to read about how easy it is to block wireguard: Let's talk about obfuscation again

 

 

Install a newer package version than available on PyPI

Written by pmd - - no comments

Try to simply update

pi@raspberrypi:~ $ sudo python3 -m pip install --upgrade mplfinance --no-cache-dir
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already up-to-date: mplfinance in /usr/local/lib/python3.7/dist-packages (0.11.0)
Requirement already satisfied, skipping upgrade: matplotlib in /usr/lib/python3/dist-packages (from mplfinance) (3.0.2)
pi@raspberrypi:~ $

It says the package is already at the newer version 0.11, but on github.com/matplotlib/mplfinance there is even newer.

Alternative that worked

Download the sources :

pi@raspberrypi:~ $ wget https://github.com/matplotlib/mplfinance/archive/master.zip
--2020-08-20 21:01:12--  https://github.com/matplotlib/mplfinance/archive/master.zip
Resolving github.com (github.com)... 140.82.118.4
Connecting to github.com (github.com)|140.82.118.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/matplotlib/mplfinance/zip/master [following]
--2020-08-20 21:01:12--  https://codeload.github.com/matplotlib/mplfinance/zip/master
Resolving codeload.github.com (codeload.github.com)... 140.82.112.9
Connecting to codeload.github.com (codeload.github.com)|140.82.112.9|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘master.zip’

master.zip  [                         <=>                ]  19.37M   208KB/s    in 94s

2020-08-20 21:02:46 (212 KB/s) - ‘master.zip’ saved [20307580]

pi@raspberrypi:~ $

Unzip :

pi@raspberrypi:~ $ unzip master.zip -d mplfinance
Archive:  master.zip
3af71a860c9eb646b92a6c9d4d4ab0a129f3db79
   creating: mplfinance/mplfinance-master/
   creating: mplfinance/mplfinance-master/.github/
   creating: mplfinance/mplfinance-master/.github/ISSUE_TEMPLATE/
  inflating: mplfinance/mplfinance-master/.github/ISSUE_TEMPLATE/ask-a-question.md
  inflating: mplfinance/mplfinance-master/.github/ISSUE_TEMPLATE/bug_report.md
[... many lines ...]
finishing deferred symbolic links:
  mplfinance/mplfinance-master/examples/original_flavor/data -> ../data
pi@raspberrypi:~ $

Go in unziped folder :

pi@raspberrypi:~ $ cd mplfinance/mplfinance-master/
pi@raspberrypi:~/mplfinance/mplfinance-master $

Install :

pi@raspberrypi:~/mplfinance/mplfinance-master $ sudo python3 -m pip install .
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/pi/mplfinance/mplfinance-master
Requirement already satisfied: matplotlib in /usr/lib/python3/dist-packages (from mplfinance==0.12.7a1) (3.0.2)
Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from mplfinance==0.12.7a1) (1.0.3)
Requirement already satisfied: python-dateutil>=2.6.1 in /usr/lib/python3/dist-packages (from pandas->mplfinance==0.12.7a1) (2.7.3)
Requirement already satisfied: pytz>=2017.2 in /usr/lib/python3/dist-packages (from pandas->mplfinance==0.12.7a1) (2019.1)
Requirement already satisfied: numpy>=1.13.3 in /usr/lib/python3/dist-packages (from pandas->mplfinance==0.12.7a1) (1.16.2)
Building wheels for collected packages: mplfinance
  Running setup.py bdist_wheel for mplfinance ... done
  Stored in directory: /root/.cache/pip/wheels/4e/71/07/c9cc7215e05dd2bcc76f171eaf646c5e069e3bc296fb8defb9
Successfully built mplfinance
Installing collected packages: mplfinance
  Found existing installation: mplfinance 0.11.0
    Uninstalling mplfinance-0.11.0:
      Successfully uninstalled mplfinance-0.11.0
Successfully installed mplfinance-0.12.7a1
pi@raspberrypi:~/mplfinance/mplfinance-master $

Done, should be good.

Source

get_throttled monitoring in Munin

Written by pmd - - no comments

Install Munin

https://angristan.fr/monitorer-serveur-linux-munin/

Make a new pluggin

$ sudo nano /usr/share/munin/plugins/getthrottled
#!/bin/sh
#
# Munin plugin for measuring the core temperature of the BCM2835 SoC on a
# Raspberry Pi.
# https://github.com/munin-monitoring/contrib/blob/master/plugins/raspberry-pi/raspi_temp
#
# $ vcgencmd get_throttled
# throttled=0x50000
#
# 111100000000000001010
# ||||             ||||_ under-voltage
# ||||             |||_ currently throttled
# ||||             ||_ arm frequency capped
# ||||             |_ soft temperature reached
# ||||_ under-voltage has occurred since last reboot
# |||_ throttling has occurred since last reboot
# ||_ arm frequency capped has occurred since last reboot
# |_ soft temperature reached since last reboot
#
#
#
#
case $1 in
    config)
        cat <<'EOM'
graph_title Core Throttled
graph_vlabel Core has already throttled since reboot
graph_category sensors
graph_args --base 1000 -l 0
getthrottled.label Throttled since start
getthrottled2.label Currently throttled
EOM
        exit 0;;
esac

echo "getthrottled2.value $(($(/opt/vc/bin/vcgencmd get_throttled | cut -c17-17)))"
echo "getthrottled.value $(($(/opt/vc/bin/vcgencmd get_throttled | cut -c13-14)))"

Then need to make it execuable :

$ sudo chmod +x /usr/share/munin/plugins/getthrottled

Test it as munin user :

$ sudo munin-run getthrottled

If you get that :

$ sudo munin-run getthrottled
getthrottled2.value i
getthrottled.value iz
$ sudo -u munin vcgencmd getthrottled
VCHI initialization failed

Somehow you will need to add munin user in the video group :

sudo usermod -a -G video munin

Or maybe that was that helped :

$ sudo nano /etc/munin/plugin-conf.d/getthrottled
[getthrottled]
user root

Then it will work :

$ sudo munin-run getthrottled
getthrottled2.value 2
getthrottled.value 20

Restart what is necessary :

$ sudo service munin-node restart

Source : http://guide.munin-monitoring.org/en/latest/develop/plugins/howto-write-plugins.html

Python 3.7 + Selenium on Raspberry Pi 3 and on Windows 10

Written by pmd - - no comments

Raspian buster

$ sudo apt-get install chromium-chromedriver xvfb
$ sudo python3 -m pip install pyvirtualdisplay selenium

Windows 10

Download Chrome. Install it.

https://chromedriver.chromium.org/getting-started

https://stackoverflow.com/questions/33150351/how-do-i-install-chromedriver-on-windows-10-and-run-selenium-tests-with-chrome

 

Python3

#!/usr/bin/python3
# -*-coding:Utf-8 -*

# Selenium
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

mg = 'tck_000003X1'
options = Options()
options.add_experimental_option("prefs", {
    #"download.default_directory": default_download_directory,
    "download.prompt_for_download": False,
    "download.directory_upgrade": True,
    "safebrowsing.enabled": True,  
})
options.add_experimental_option("excludeSwitches", ["enable-logging"])

browser = webdriver.Chrome(options=options, executable_path=r'C:\Windows\chromedriver.exe')

url = "http://ms.com/ms.html?t=" + mg
browser.get(url)
etoiles = browser.find_element_by_id("etoile_span").get_attribute('class')
print("Nombre d'étoile : " + etoiles[-1])

browser.quit()

Python on raspberry

Written by pmd - - no comments

Begin

Installer et mettre à jour Python

Python 2 and Python 3 sont préinstallés sur Raspbian, mais au cas ou pour installer python3 :

sudo apt-get install python3

Ecrire un programme en python

Créer un fichier :

nano hello-world.py

Le remplir avec :

#!/usr/bin/python3

print("Hello, World!")

CTRL+X then Y.

Tous les programmes en Python doivent être avec l'extension ".py".

Executer un programme en Python

python3 hello-world.py

Rendre un programme python executable

Pour rendre un fichier ".py" executable :

chmod +x file-name.py

Puis pour l'executer :

./file-name.py

Source

Parse HTML : Beautiful Soup

Installing :

sudo apt-get install python-bs4 python3-bs4

Premier script

#!/usr/bin/python
from bs4 import BeautifulSoup, Comment

print("Hello, World!");

# save as local file 'webpage.html'
import urllib.request;
#urllib.request.urlretrieve ("https://www.xe.com/fr/currencytables/?from=EUR&date=2019-07-11", "webpage.html");

# read entire file and close immediately after block ends
with open('webpage.html', 'r') as f:
    html_doc = f.read()
#print(html_doc);

# on ouvre la page web téléchargée
soup = BeautifulSoup(html_doc, 'lxml')
# on vire tous les commentaires html
for element in soup(text=lambda text: isinstance(text, Comment)):
    element.extract()
# on vire tous les liens
for a in soup.findAll('a'):
    a.replaceWithChildren()

#print(soup.prettify())
all_rates = soup.tbody.find_all("tr")

# Loop over all elements of a list
for element_tr in all_rates:
    print(element_tr)

Installation de modules avec pip

Sous Windows pour installer des modules :

C:\Users\<username>\AppData\Local\Programs\Python\Python37\Scripts>pip3.7.exe install lxml

Sous linux, souvent il y a des paquets déjà prêts. Sinon :

sudo pip install pandas

Faire des choses différentes si script sous Windows ou sous Linux

import platform
if platform.system() == 'Windows':
    #Windows
    work_folder = Path('C:\Users\...')
elif platform.system() == 'Linux':
    #Linux
    work_folder = Path('/home/user/...')

Start script from Notepad++

Aller dans Execution > Executer et utiliser cette ligne :

cmd /k "python.exe $(FULL_CURRENT_PATH)"

Very usefull for yfinance

Fixes #192 - If no _institutial_holders is found by JeremyRitchie · Pull Request #196 · ranaroussi/yfinance · GitHub

*.py vers *.exe

Installer auto-py-to-exe :

pip install auto-py-to-exe

Lancer le programme depuis le bon dossier :

auto-py-to-exe

Source

 

CSV editor

Written by pmd - - no comments

This script is working under Windows 7 and Windows 10 using Busybox.
To start it I made a shortcut to "busybox.exe sh -l".

To edit CSV for better and easier human reading.

#!/bin/bash
# File to be modified
FILE_IN="$1"
# Output file to be generated
FILE_OUT=$(echo -n "$FILE_IN" | sed 's@.csv@_mod.csv@g')
cp $FILE_IN $FILE_OUT

# delete all empty lines after header (header is 11 lines long)
sed -i '12,${\@^$@d;}' $FILE_OUT

# put back all samples together (delete in-file headers except first one)
sed -i '12,${\@Arming date:@{N;N;d}}' $FILE_OUT

# replace long path by short path of variables
# before: CPU1/CPU_fast//CPU_fast/appli/test/sf_12/do_low_power_tests/n_lpt
# after: CPU1/n_lpt
LINE_TO_MODIFY=$(cat $FILE_OUT | grep "/")
LINE_MODIFIED=$(echo -n "$LINE_TO_MODIFY" | sed -e 's@/[^;]*/@/@g')
echo "Old line to modify:"
echo "$LINE_TO_MODIFY"
echo
echo "New line modified:"
echo "$LINE_MODIFIED"

# remove and replace variable names with short path
sed -i "s@$LINE_TO_MODIFY@$LINE_MODIFIED@g" $FILE_OUT

dhcpcd and dhcpcd-run-hooks on Raspbian

Written by pmd - - no comments

If you are using Raspbian GNU/Linux 9 (stretch), then you are most likely using dhcpcd as your DHCP and DHCPv6 client. How to know your OS version? Here :

$ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"
NAME="Raspbian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
  • dhcpcd gets the host information (IP address, routes, etc) from a DHCP server and configures the network interface of the machine on which it is running. Manual of DHCPCD(8).
  • dhcpcd-run-hooks is a DHCP client configuration script. It is used by dhcpcd to run any system and user defined hook scripts. Manual of DHCPCD-RUN-HOOKS(8).

dhcpcd

It seems that to restart all network interfaces this works :

$ sudo systemctl restart dhcpcd.service

To understand what happened, better read dhcpcd logs, here is what we can see:

$ journalctl -u dhcpcd
Dec 21 15:38:04 raspberrypi systemd[1]: Stopping dhcpcd on all interfaces...
Dec 21 15:38:04 raspberrypi dhcpcd[15174]: sending signal TERM to pid 14888
Dec 21 15:38:04 raspberrypi dhcpcd[15174]: waiting for pid 14888 to exit
Dec 21 15:38:04 raspberrypi dhcpcd[14888]: received SIGTERM, stopping
Dec 21 15:38:04 raspberrypi dhcpcd[14888]: eth0: removing interface
Dec 21 15:38:04 raspberrypi dhcpcd[14888]: wlan0: removing interface
Dec 21 15:38:04 raspberrypi dhcpcd[15174]: sending signal TERM to pid 14888
Dec 21 15:38:04 raspberrypi dhcpcd[15174]: waiting for pid 14888 to exit
Dec 21 15:38:05 raspberrypi systemd[1]: Stopped dhcpcd on all interfaces.
Dec 21 15:38:05 raspberrypi systemd[1]: Starting dhcpcd on all interfaces...
Dec 21 15:38:05 raspberrypi dhcpcd[15180]: dev: loaded udev
Dec 21 15:38:05 raspberrypi systemd[1]: Started dhcpcd on all interfaces.
Dec 21 15:38:05 raspberrypi dhcpcd-run-hooks[15203]: wlan0: starting wpa_supplicant
Dec 21 15:38:05 raspberrypi dhcpcd[15185]: eth0: waiting for carrier
Dec 21 15:38:05 raspberrypi dhcpcd[15185]: wlan0: waiting for carrier
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: carrier acquired
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: DUID 00:01:00:01:21:b0:de:63:b8:27:eb:4d:6c:87
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: IAID eb:5d:6d:97
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: adding address fe80::2882:e851:cf0c:9242
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: soliciting an IPv6 router
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: rebinding lease of 192.168.254.200
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: leased 192.168.254.200 for 86400 seconds
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: adding route to 192.168.254.0/24
Dec 21 15:38:06 raspberrypi dhcpcd[15185]: wlan0: adding default route via 192.168.254.1
Dec 21 15:38:19 raspberrypi dhcpcd[15185]: wlan0: no IPv6 Routers available

Source

dhcpcd-run-hooks

dhcpcd runs /lib/dhcpcd/dhcpcd-run-hooks. This script runs each script found in /lib/dhcpcd/dhcpcd-hooks in a lexical order.
dhcpcd-run-hooks is used by dhcpcd to run any system and user defined hook scripts. System hook scripts are found in /lib/dhcpcd/dhcpcd-hooks.
There is a list of reasons why dhcpcd-run-hooks could be invoked, such as BOUND (= dhcpcd obtained a new lease from a DHCP server = computer has an IP assigned on the interface).
Many variables will then be set, along with any protocol supplied ones, such as :

  • $interface : the name of the interface
  • $reason : reason why dhcpcd-run-hooks was invoked
  • $if_up : true if the interface is up, otherwise false.

To know what is the current IPv4 address after an interface BOUNDed dhcpcd obtained a new lease from a DHCP server.:

ifconfig $interface | grep "inet " | awk '{print $2}'

I created /lib/dhcpcd/dhcpcd-hooks/60-test_by_pmd in order to play with this functionality. Shell script I used in order to check IP after new DHCP lease was obtained (writing output in /home/pi/test.txt in order to see what happened afterward):

if [ "$reason" = "BOUND" ] && [ "$if_up" = "true" ]; then
     echo "$(date +"%d-%m-%y %H:%M:%S ") $reason $interface IP:$if_up DOWN:$if_down" >> /home/pi/test.txt
     ifconfig $interface | grep "inet " | awk '{print $2}' >> /home/pi/test.txt
fi

Sources : both manuals.

AUTO-Update of openvpn configuration

Written by pmd - - no comments

It happens that NordVPN openvpn configuration files don't work anymore after a while.

I made a bash script to update the configuration easy and fast:

#!/bin/bash
# Mise a jour de la configuration NordVPN
# Dossier temporaire
DOSSIER_TEMP="/home/pi/NordVPNautoUpdate"
#Dossier des configurations openvpn
DOSSIER_OPENVPN="/etc/openvpn"
CONF_OPENVPN="server.conf"
AUTOLOGIN_OPENVPN="login.txt"
# Pays
PAYS="ua"
# TCP ou UDP?
PROTO="udp"
# Archive de configuration NordVPN
ARCHIVE_NORDVPN="https://downloads.nordcdn.com/configs/archives/servers/ovpn.zip"
FICHIER=$(echo -n $ARCHIVE_NORDVPN | awk -F "/" '{printf $NF}')

# On supprime et on crée le dossier temp
rm -rf $DOSSIER_TEMP
mkdir $DOSSIER_TEMP

# On télécharge tous les fichiers de conf NordVPN
wget $ARCHIVE_NORDVPN -P $DOSSIER_TEMP

# On dézip l'archive téléchargée
mkdir $DOSSIER_TEMP/temp
unzip -q "$DOSSIER_TEMP/$FICHIER" -d $DOSSIER_TEMP/temp

# On garde que les conf d'un certain pays
mv $DOSSIER_TEMP/temp/ovpn_$PROTO/$PAYS[0-9]*$PROTO* $DOSSIER_TEMP/
rm -r $DOSSIER_TEMP/temp

# On supprime les conf double-VPN et l'archive zip
NbConf=0
for ConfOpenVPN in $DOSSIER_TEMP/*; do
        if [[ $ConfOpenVPN != */$PAYS[0-9]*$PROTO* ]]; then
                rm $ConfOpenVPN
        else
                # On compte le nombre de conf restantes
                NbConf=$((NbConf+1))
        fi
done

# On choisi une conf au hasard
NbConf=$((1 + RANDOM % $NbConf))
COMPTEUR=0
for ConfOpenVPN in $DOSSIER_TEMP/*; do
        COMPTEUR=$((COMPTEUR+1))
        if [[ $COMPTEUR -eq $NbConf ]]; then
            # On modifie la configuration pour authentification automatique
            sed -i "s@auth-user-pass@auth-user-pass $DOSSIER_OPENVPN/$AUTOLOGIN_OPENVPN@" $ConfOpenVPN
            # On déplace la conf dans le dossier openvpn
            sudo cp -f $ConfOpenVPN $DOSSIER_OPENVPN
            sudo cp -f $ConfOpenVPN $DOSSIER_OPENVPN/$CONF_OPENVPN
                        # On informe
                        echo "Configuration installée :"
                        echo $ConfOpenVPN | awk -F "/" '{printf $NF}'
                        echo ""
        fi
done

# On supprime le dossier temporaire
rm -rf $DOSSIER_TEMP
# On redémarre openvpn avec la nouvelle configuration
sudo service openvpn restart

Then simply execute the script:

bash NordVPNautoUpdate.sh

Unlock standalone Airsonic database

Written by pmd - - no comments

If your airsonic is stucked for some reasons such as Liquibase Update Failed and Waiting for changelog lock...

liquibase.exception.LockException: Could not acquire change log lock. Currently locked by 192.168.254.200 (192.168.254.200) since 18/11/18 16:39
  1. Stop Airsonic for sure :
    sudo service airsonic stop
  2. Read logs /var/airsonic/airsonic.log :
    nano /var/airsonic/airsonic.log
  3. Backup your database folder /var/airsonic/db :
    sudo cp /var/airsonic/db /var/airsonic/db_backup
  4. Edit /var/airsonic/db/airsonic.script :
    sudo nano /var/airsonic/db/airsonic.script
    • remove all lines where you can find DATABASECHANGELOGLOCK by example :
      • CREATE MEMORY TABLE DATABASECHANGELOGLOCK.....
      • INSERT INTO DATABASECHANGELOGLOCK VALUES(1,TRUE,'2018-11-18 16:39:56.266000000','192.168.254.200 (192.168.254.200)')
  5. Save file.
  6. Start Airsonic :
    sudo service airsonic start
  7. Check Airsonic status :
    sudo service airsonic status

If you wanna reset your Airsonic music server, just delete the database /var/airsonic/db ...

Source 1, 2, 3.

Update Airsonic on Raspberry Pi 3

Written by pmd - - no comments

How to update a standalone Airsonic

  • Stop the current running Airsonic:
$ sudo systemctl stop airsonic.service
$ cd /var/airsonic/
$ sudo mv airsonic.war airsonic.war.10.1.1.bak
$ sudo wget https://github.com/airsonic/airsonic/releases/download/v10.1.2/airsonic.war --output-document=/var/airsonic/airsonic.war
  • Put good rights to all folder (new airsonic.war is not owned by airsonic), and verify it has been applied:
$ sudo chown -R airsonic:airsonic /var/airsonic
$ ls
total 151M
-rw-r--r--  1 airsonic airsonic 760K Oct 11 19:52 airsonic.log
-rw-r--r--  1 airsonic airsonic  11M Oct  3 00:20 airsonic.log.1
-rw-r--r--  1 airsonic airsonic  978 Oct 11 04:00 airsonic.properties
-rw-r--r--  1 airsonic airsonic  70M Jul 28 22:10 airsonic.war
-rw-r--r--  1 airsonic airsonic  70M Dec 16  2017 airsonic.war.10.1.1.bak
drwxr-xr-x  2 airsonic airsonic 4.0K Oct 11 19:49 db
drwxr-xr-x  2 airsonic airsonic  20K Sep 14 16:08 lastfmcache
drwxr-xr-x  7 airsonic airsonic 4.0K Jul 22 10:36 lucene2
-rw-r--r--  1 airsonic airsonic  741 Oct 11 19:50 rollback.sql
drwxr-xr-x 16 airsonic airsonic 4.0K Oct  4 17:45 thumbs
drwxr-xr-x  2 airsonic airsonic 4.0K Jul 22 10:54 transcode
  • Start updated Airsonic and restart proxy server:
$ sudo systemctl start airsonic.service
$ sudo service nginx restart

Nothing too hard, basically, you need to replace the airsonic.war by the updated one.

Rss feed of the category