VPN tunnels through HTTP proxy using SSH

The title of this post is beatufill: 7 words, 3 acronyms equally distributed composed of 3,4 and 3 letters.
But that’s not the topic….instead, today, we will learn how to setup a VPN tunnel using SSH when you are behind proxy.

You will need:
– a first tool: sshuttle
– an SSH client able to receive and process the ProxyCommand directive
– a remote SSH server running on port 443 or 80
– another tool: corkscrew
– a proxy server only allowing HTTP(S) traffic

Let’s describe each in reverse order

Proxy Server

You should not have too much control on it, but if the proxy server requires authentication you should get your pair of credentials. Also, if using corkscrew like here, the proxy must supports CONNECT command, otherwise, you should use httptunnel instead.

Corkscrew

corkscrew is a simple tool to tunnel TCP connections through an HTTP proxy supporting the CONNECT method. It reads stdin and writes to std- out during the connection, just like nectat.
We will use it to connect to an SSH server running on a remote 443 port through the HTTPS proxy. To do so, we will need to set corkscrew as the ProxyCommand for our SSH client. If your proxy requires authentication, you have to set the credentials in a separate file, lets say ~/.ssh/corkscrew-authfile with the patten username:password

my_proxy_unsername:my_p@ssword

SSH Server

A raspberry Pi hidden at home or even an AWS Free Tier machine should be sufficient. The required configuration parameter needs the following:

# What ports, IPs and protocols we listen for
Port 22
Port 443

# Authentication:
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile	%h/.ssh/authorized_keys

SSH Client

The SSH client configuration will be setup in your .ssh/config file, so you don’t need to type it every time you want to use your tunnel.

Host my-remote-ssh-server.mydomain.com
  ProxyCommand corkscrew http-nasty-proxy.mycompany.com 8080 my-remote-ssh-server.mydomain.com 443  /Users/Akram/.ssh/corkscrew-authfile

Then, every time you will do:

ssh my-remote-ssh-server.mydomain.com

You will be automagically connected to your SSH box, because the SSH client will delegate its connexion management to corkscrew that will connect to http-nasty-proxy.mycompany.com on port 8080 using the credentials in file /Users/Akram/.ssh/corkscrew-authfile and then convert the SSH commands into HTTP+CONNECT request going to my-remote-ssh-server.mydomain.com on port 443.

That was the most difficult part. Once you are connected to your SSH box, the world is then open to you!

sshuttle

sshuttle is the ultimate tool that we will use: It is a transparent VPN proxy through SSH. sshuttle documentation describes briefly the way it works and gives many example of usages. The one that I uses if simply this command line:

sshuttle  --dns -r user@my-remote-ssh-server.mydomain.com 0/0

Juste not here that my-remote-ssh-server.mydomain.com is the address of the server for which you have setup ProxyCommand configuration. Since sshuttle will use SSH under the cover, you have made the sufficient work to make the connection work (even through HTTPS Proxy).
In my case, I added the –dns option to also allow DNS traffic to go through my tunnel because corporate DNS traffic is blocked.
If the connection succeeds, you will see a message “client connected”.
Et voilà….all your connections will go through sshuttle to reach the internet

OpenShift 3 cheatsheet

Here are a few useful commands that you may very often use on OpenShift 3.

Mark a node as non schedulable: Useful once you’ve created OpenShift router and registry to avoid any other scheduling on these nodes:

oadm manage-node node1.example.com --schedulable=false

Deploy OpenShift integrated docker registry:

 oadm registry --config=/etc/openshift/master/admin.kubeconfig \
    --credentials=/etc/openshift/master/openshift-registry.kubeconfig \
    --images='registry.access.redhat.com/openshift3/ose-${component}:${version}' 

Deploy an OpenShift router:

oadm router myrouter --replicas= \
    --credentials='/etc/openshift/master/openshift-router.kubeconfig' \
    --service-account=router

Adding/setting insecure-registry to docker machine afterwards

Running docker on non-Linux based environment became very convenient and easy with docker-machine which is the successor of docker-boot.

Basically, docker-machine allows you to manage multiple virtual machines running Linux to host your docker installation and then allows you to run your containers.
More than a fantastic tool for OSX and Windows, it is also a very clever and practical way to develop multiple container images or several applications (for different project for examples) using containers.

If you want your docker-machine to use an your own in-house registry or any other, it is not a big issue, until the registry uses HTTPS, and in most of the cases you will get the following error:

docker tag -f my-app/my-app-server:v1.0.14-25-gfefb196 dockerhub.rnd.mycompany.net:5000/my-app/my-app-server:v1.0.14-25-gfefb196
docker push dockerhub.rnd.mycompany.net:5000/my-app/my-app-server:v1.0.14-25-gfefb196
The push refers to a repository [dockerhub.rnd.mycompany.net:5000/my-app/my-app-server] (len: 1)
unable to ping registry endpoint https://dockerhub.rnd.mycompany.net:5000/v0/
v2 ping attempt failed with error: Get https://dockerhub.rnd.mycompany.net:5000/v2/: x509: certificate signed by unknown authority
 v1 ping attempt failed
 with error: Get https://dockerhub.rnd.mycompany.net:5000/v1/_ping: x509: certificate signed by unknown authority

For this case, docker-machine has a fantastic option which is available on creation of a machine:

docker-machine create --driver virtualbox --engine-insecure-registry myregistry:5000 mycompany

But, suppose that you want to add another registry once your docker-machine is created: Unfortunately, I can’t find an option yet to edit the existing configuration of a VM.
You will have to edit your configuration file which is located on your host system (your OSX or Windows home) and add it manually:

vim  ~/.docker/machine/machines/mycompany/config.json

Then, you’ll have to edit the config.json file and locate the array named:InsecureRegistry and simply append an element on it.
It should looks like this:

{
  "ConfigVersion": 1,
 // Truncated for readability 
  "DriverName": "virtualbox",
  "HostOptions": {
    "Driver": "",
    "Memory": 0,
    "Disk": 0,
    "EngineOptions": {
      "ArbitraryFlags": [],
      "Dns": null,
      "GraphDir": "",
      "Env": [],
      "Ipv6": false,
      "InsecureRegistry": [
        "dockerhub.rnd.mycompany.net:5002",
        "dockerhub.rnd.mycompany.net:5000",
        "dockerhub.rnd.mycompany.net:5001"
      ],
      // Truncated for readability 
  },
  "StorePath": "/Users/Akram/.docker/machine/machines/mycompany"
}

OpenShift cheat sheet for beginners

Here is a simple cheatsheet for OpenShift beginners that will help you to visualise some basic settings about your projects, applications, pods in order to debug or get informations about how they behave.

Listing all your projects

oc get projects

This will give you the list of all the project that you can work on an highlight the current project.

Positioning the current project

oc project my-project

This will switch you current project to my-project. This settings is save in your ~/.kube/config file, so if multiple persons are using oc simultaneously with the same user, just mind no overriding each other.

Listing the existing pods (applications)

oc get pods

This will list all the pods (a wrapper for containers, even if generally 1 pod = 1 container) and show you status for each of them.

Checking status for pod

oc describe pod 

This will display information about the pod lifecycle: the node on which it has been scheduled, the status of the docker image on the node (image existing or pulling or failed to be pulled), the readiness and liveness status, and if the pod is started or stopped.

Watching pods logs

oc logs -f 

The -f option is for follow, just like for the tail command. This will display the logs sent to stdout from the container.
If the pod has crashed or has stopped, it will be in a state that would prevent seeing logs unless you specify -p (for –previous) option.

oc logs -p 

Watching event on project

oc get events -w

This will show you all the OpenShift events occurring on the current project and keep watching it (-w for –watch). The evens includes scheduling events, pod startup, scheduling, etc…

Hope that this will help every beginner.

Starting with OpenShift v3 : Using the All-In-One Virtual Machine

OpenShift v3 is a PaaS management software relying on innovative technologies allowing you to run your own cloud environment in your private infrastructure, on public clouds or in hybrid way.

To get familiar with OpenShift, the best thing to do, is try to install it on your (muscled (8GB+ RAM)) laptop and deploy some of the example environment and run hello world applications on it.

To do so, I recommend you to use the All-In-One image provided by the OpenShift team at this address: http://www.openshift.org/vm/

You will have to be familiar with Vagrant and and a virtualisation tool like VirtualBox (or bitterly, a Linux Kernel based virtualisation tool like KVM) and your OpenShift 1-machine cluster will be running in minutes.

Step by step

Easy and simple, just perform the following steps. Let’s assume that we are using OSX, but the steps are very similar if using Windows or Linux of course. For convenience also, we will be using VirtualBox which is available on the 3 platforms.

Installing the tools

The required tools are: VirtualBox and Vagrant.

VirtualBox is a available on the virtual box website. Download a 5.0.x version for your platform and take a few minutes also to download the extension pack for your platform. The installation is quite straight forward by using a wizard installer: Next Next Next install.

Vagrant is a command line “script like” based tool used to control VirtualBox using command lines. The script recipe is named a Vagrantfile which will contain the whole logic for creating a Virtual Machine and settings its various configuration elements. Vagrant is also installable with OS specific packages and/or wizard based installer available from the vagrant download page.

Downloading the OpenShift All-In-One files

Visit the OpenShift All-In-One page at http://www.openshift.org/vm/ and you will see there all the materials that we are now ready to use to start our OpenShift cluster. You now all know what are the different tools referring too. So let’s continue by downloading the following elements:

  • The Vagrant Box File 1.0.6, about 2.1GB : This file is a template Virtual Box image containing the base OpenShift VM
  • The Vagrantfile : The vagrant recipe to start and run the OpenShift cluster

Once you have these files, I recommend you to put them in the same directory for example under your home directory:

mkdir -p ~/OpenShift
mv ~/Downloads/Vagrantfile ~/OpenShift
mv ~/Downloads/openshift-bootstrap-1.0.6.box ~/OpenShift

Adding the box image

To enable Vagrant to instantiate virtual machines using the provided .box image, we will have to add it to the Vagrant available boxes.

cd ~/OpenShift
vagrant box add openshift-bootstrap-1.0.6.box

Starting the VM

Before starting the VM, we will just perform a single change on the Vagrantfile, in case you have a slow laptop like mine, to avoid a timeout while starting the VM and add the following line just after config.vm.box = “openshift3”

config.vm.boot_timeout = 600

To start your VM, then simply run the following command:

vagrant up

Wait for a few minutes, and if you want to see progress, you can launch your VirtualBox console and see that a VM named “openshift3” is automatically started and configured.

Connecting to your OpenShift dashboard

Vagrant establishes a port-forward between some ports of the running VM and ports on your localhost. We may have notice this on the log messages.
The openshift master will listen on its localhost interface on port 8443 which will be mapped to your laptop localhost on port 8443 which will be convenient for OpenShift self SSL certificates to be accepted.
To connect to the dashboard, simply visit this address: https://localhost:8443/ . You will be able to see the login screen.

OpenShiftDashboardLogin

Login using the following credentials:

  • Username: admin
  • Password: admin

And you will then see the list of existing environment and applications:

OpenShiftApps

Deploying your first environment

To start a new project, simply clic on the New Project button on the upper right corner of the screen and select your environment:

OpenShiftAddProject

Running the first hello-word app

Then, you can add your first application to this project by clicking the Add to project button, and select the template. Here the EAP6 template.

OpenShiftEAPApp

 

My wordpress blog migration to OpenShift Online

After almost a year working with a custom domain on wordpress.com, the world leader platform for blogging is asking me for a renewal of my domain name and service which is quite expensive baed on the sporadic usage I do of my blog and the traffic that I have.

Anyway, that was a good opportunity to perform my blog migration to the OpenShift platform which is now very mature to host such projects and gives you the ability through the WordPress cartridge to have your on private and administrable installation of WordPress running on the cloud.

I was already running a trial version of OpenShift 2.0 online which gave me the ability to run 3 gears and I already used 2 for other private project. So this, trial instance would be perfect to host my blog.

If you are in the same situation here are some steps to follow if you want to migration a Worpdress.com blog to Openshift.

  • Create your OpenShift Online (v2) wordpress environment
  • Choose a DNS registrar which supports having CNAME on your domain level (preferable)
  • Export your last wordpress site after having install the WordPress Export plugin
  • Install your new wordpress site on OpenShift
  • Import the result of the export of the old site (images will be imported automatically, so be sure that the old site is still up, running and alive)
  • Edit your domain name in OpenShift to point to your DNS name
  • Edit your DNS zone to add a CNAME pointing to the openshift URL of your blog
  • And you are done !

 

Enable IPv6 in Pidora

I was disappointed to see that so poor documentation exists on how to enable IPv6 on a RaspberryPi running Pidora (the Fedora version for the RPi).

After hanging here and out, I finally simply managed to run it using NetworkManager configuration.
Since my RPi is a headless (no display machine), I simply installed XQuartz on my Mac laptop and enabled X11 Forwarding on my SSH session.

By default, the sshd server on RPi does not enable it, so you have to add the following lines to /etc/ssh/sshd_config:

X11Forwarding yes
X11UseLocalhost no

Then restart the service:

# service sshd restart

And on your laptop connect by ssh with the following options:

ssh -Y root@pidora.local

Then launch nm-connection-editor and edit eth0 network connection and go to IPv6 tab. Simply select the configuration to Automatic instead of Ignore which is default.

# nm-connection-editor

nm1 nm2

 

Locked out of your Mac : A few tips and tools

I was locked out of my mac for a stupid reason: I installed and downloaded the VPN Server Enabler, and when configuring a user to connect, I chose my own user.
The bad thing is that VPN Server Enabler changes the shell to false and the home to some private empty dir. If you set a password for the VPN user it will also change your own user’s password….ha ha

Well, the surprise happened this morning when trying to login while in the plane. After being logged in, I was automatically redirect to the Locked Screen asking my password again and again.

I rapidly understood that something changed with my user.

Single User Mode: cmd + S

It is documented everywhere, so the first thing to do is to restart your computer and hold cmd + S key while logging in. It is supposed to give you a unix shell, but for me it did not: My hard disk is encrypted and I only realised that the login screen that OSX was presenting me after the cmd+s boot sequence was to type the encryption key.
So first tip, if your disk is encrypted, you will have to type your encryption passphrase before gaining access to a single user shell.

Reading my users details

Here, you will have to use the dscl command. To show your user info, which are quite a large file, because it also contains the base64 encoding of your JPEG avatar, just type:
dscl . read /Users/YourUser

After some lines of text, you will see something like

RecordType: dsRecTypeStandard:Users
UniqueID: 501
UserShell: /bin/false

Bingo, my shell has been changed. Restoring it to a more viable thing requires the use of the chsh command:

chsh MyUser -s /bin/bash 

And then, everything was fine I was able to login again.

A few more changes to users

Returning back to my OSX, I created a rescue user just in case. It’s like letting the rescue keys to your relatives. Always a good idea.

And then, I realised that MyUser was much more altered: The home directory and the full name was changed. Easy here, the Users & Groups menus from OSX allows changing this by “right” clicking on the user and selecting Advanced Options

Hope This Helps

Posted in Mac