Author: jonas

Posted on

Ansible with Dell PowerEdge servers

Automate everything and have more time left for coffee and ridiculously-sized donuts! PowerEdge servers and Ansible automation is a match made in silicon heaven (just ask Kryten!) Included are six videos covering everything from the ground up.

Installation steps for Ansible

To be used with the first video: The installation steps for Ansible as well as the OpenManage modules for PowerEdge can be downloaded from here: link

Posted on

dnsmasq: failed to create listening socket for port 53: Address already in use

Ubuntu 19:10: systemd-resolved blocks port 53 and thereby preventing any service using port 53 (like dnsmasq) from starting

 Jan 29 03:31:58 ubuntupxe02 dnsmasq[2386]: dnsmasq: failed to create listening socket for port 53: Address already in use
 Jan 29 03:31:58 ubuntupxe02 dnsmasq[2386]: failed to create listening socket for port 53: Address already in use
 Jan 29 03:31:58 ubuntupxe02 dnsmasq[2386]: FAILED to start up
 Jan 29 03:31:58 ubuntupxe02 systemd[1]: dnsmasq.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
 Jan 29 03:31:58 ubuntupxe02 systemd[1]: dnsmasq.service: Failed with result 'exit-code'.
 Jan 29 03:31:58 ubuntupxe02 systemd[1]: Failed to start dnsmasq - A lightweight DHCP and caching DNS server.

Verify that the port is used by systemd-resolve

jonas@ubuntupxe02:~$ sudo lsof -i -P -n | grep LIST
 systemd-r  784 systemd-resolve   13u  IPv4  19378      0t0  TCP 127.0.0.53:53 (LISTEN)
 sshd       859            root    3u  IPv4  23918      0t0  TCP *:22 (LISTEN)
 sshd       859            root    4u  IPv6  23920      0t0  TCP *:22 (LISTEN)
 apache2   1705            root    4u  IPv6  27900      0t0  TCP *:80 (LISTEN)
 apache2   1706        www-data    4u  IPv6  27900      0t0  TCP *:80 (LISTEN)
 apache2   1707        www-data    4u  IPv6  27900      0t0  TCP *:80 (LISTEN)

Stop the service

jonas@ubuntupxe02:~$ sudo systemctl stop systemd-resolved

Edit the systemd-resolved config file

 jonas@ubuntupxe02:~$ sudo vi /etc/systemd/resolved.conf
 jonas@ubuntupxe02:~$ cat !$ | grep DNS
 cat /etc/systemd/resolved.conf | grep DNS
 DNS=8.8.8.8
 FallbackDNS=
 MulticastDNS=no
 DNSSEC=no
 DNSOverTLS=no
 DNSStubListener=no
 jonas@ubuntupxe02:~$

Create symlink to /etc/resolv.conf

jonas@ubuntupxe02:~$ sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf

Start systemd-resolved service

jonas@ubuntupxe02:~$ sudo systemctl start systemd-resolved

Start dnsmasq

jonas@ubuntupxe02:~$ sudo systemctl start dnsmasq
jonas@ubuntupxe02:~$
jonas@ubuntupxe02:~$ sudo systemctl status dnsmasq
 ● dnsmasq.service - dnsmasq - A lightweight DHCP and caching DNS server
    Loaded: loaded (/lib/systemd/system/dnsmasq.service; enabled; vendor preset: enabled)
    Active: active (running) since Wed 2020-01-29 03:56:12 UTC; 6s ago
   Process: 1312 ExecStartPre=/usr/sbin/dnsmasq --test (code=exited, status=0/SUCCESS)
   Process: 1314 ExecStart=/etc/init.d/dnsmasq systemd-exec (code=exited, status=0/SUCCESS)

Verify that dnsmasq is now the user of port 53

jonas@ubuntupxe02:~$ sudo lsof -i -P -n | grep LISTEN
 sshd     823     root    3u  IPv4  23016      0t0  TCP *:22 (LISTEN)
 sshd     823     root    4u  IPv6  23018      0t0  TCP *:22 (LISTEN)
 apache2  874     root    4u  IPv6  22454      0t0  TCP *:80 (LISTEN)
 apache2  875 www-data    4u  IPv6  22454      0t0  TCP *:80 (LISTEN)
 apache2  876 www-data    4u  IPv6  22454      0t0  TCP *:80 (LISTEN)
 dnsmasq 1331  dnsmasq    5u  IPv4  28097      0t0  TCP *:53 (LISTEN)
 dnsmasq 1331  dnsmasq    7u  IPv6  28099      0t0  TCP *:53 (LISTEN)

Kudos to Nitin Gurbani for the solution

Posted on

Kubernetes home lab: Upgraded edition with functional LoadBalancer and external access to pods

In a previous post we covered the deployment of a home k8s lab, but this post will show a much better way to do it as well as improving on the end result – a fully functional local cluster.

The installation is done using Vagrant with Flannel networking and MetalLB for load balancing.

Commands for the session can be downloaded from here: https://jonamiki.com/wp-content/uploads/2019/11/kubectl-demo-commands.txt

Background

Why go through the trouble of setting up a home lab for k8s? Well, while using public cloud services is a quick and easy way it will cost money to deploy and run. It will also rely upon predefined cloud formation templates which have already been created. Doing it locally can provide both a more economical way to use k8s as well as give more insight into the internal workings and how it’s actually set up.

Why not use Minikube? Because it’s overly simplified. Using a cluster deployment like this is not only a better learning and testing experience but it also provides an overall more realistic experience of a “real” k8s installation.

Configuration files

Download the Vagrant, Flannel and MetalLB files from GitHub or clone with Git

git clone https://github.com/jonas-werner/k8s-home-lab-with-vagrant.git

Video: Editing the config files and standing up the cluster

Video: Getting started using the new K8s cluster

Enjoy your new Kubernetes powers!

Posted on

Ubuntu: The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY

Error when adding repo to install kubectl

jonas@octo:~$ sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
Hit:1 http://jp.archive.ubuntu.com/ubuntu bionic InRelease
Get:2 https://nvidia.github.io/libnvidia-container/ubuntu18.04/amd64  InRelease [1,106 B]                                                                                                  
Get:3 https://nvidia.github.io/nvidia-container-runtime/ubuntu18.04/amd64  InRelease [1,103 B]                                                                                             
Hit:4 http://jp.archive.ubuntu.com/ubuntu bionic-updates InRelease                                                                                                                         
Hit:5 https://download.docker.com/linux/ubuntu bionic InRelease                                                                                                                            
Get:6 https://nvidia.github.io/nvidia-docker/ubuntu18.04/amd64  InRelease [1,096 B]                                                                                                        
Hit:7 http://jp.archive.ubuntu.com/ubuntu bionic-backports InRelease                                                                                                                       
Hit:9 http://security.ubuntu.com/ubuntu bionic-security InRelease                                                                                                                 
Get:8 https://packages.cloud.google.com/apt kubernetes-xenial InRelease [8,993 B]                                                                   
Err:8 https://packages.cloud.google.com/apt kubernetes-xenial InRelease                              
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 6A030B21BA07F4FB
Hit:10 https://cf-cli-debian-repo.s3.amazonaws.com stable InRelease                             
Reading package lists… Done                                        
W: GPG error: https://packages.cloud.google.com/apt kubernetes-xenial InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 6A030B21BA07F4FB
E: The repository 'http://apt.kubernetes.io kubernetes-xenial InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

Solved by adding the missing key as follows

jonas@octo:~$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 6A030B21BA07F4FB

Executing: /tmp/apt-key-gpghome.JKpxFjtwsU/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys 6A030B21BA07F4FB
gpg: key 6A030B21BA07F4FB: public key "Google Cloud Packages Automatic Signing Key <a href="mailto:gc-team@google.com">gc-team@google.com</a>" imported
gpg: Total number processed: 1
gpg:               imported: 1
jonas@octo:~$
Posted on

Voice controlled Docker container deployment system using AWS and a Raspberry Pi

Playing around with AWS Lambda, Rekognition, Polly, DynamoDB, Lex, S3, etc. to create a system for deploying Docker containers by talking to a Raspberry Pi. The containers are deployed locally on a PC running the “p4docker” service while the other two services (p4security and p4voiceui) are running on the Raspberry Pi.

This was part of a project for an internal Pied Piper course here at Dell Tech earlier this year: https://bigdatadownunder.com/2019/10/11/innovating-ground-up-project-piper/

The code can be found here: