Zhixian's Tech Blog

2017-07-12

Using ACMESharp to get SSL certificates from Let’s Encrypt

This blog post is a reminder note to myself on how to use the ACMESharp PowerShell module to get SSL certificates from Let’s Encrypt CA.

Essentially, the usage can be divided into the following phases:

  1. Install ACMESharp PowerShell module
  2. Import ACMESharp PowerShell module
  3. Initial (one-time) setup
  4. Register DNS of certificate
  5. Get “challenge” details (to prove that you are the owner of the domain)
  6. Signal Let’s Encrypt to confirm your challenge answer
  7. Download certificates

Steps 1-3 is only for setting up on a new PC.
Step 2, 4 should be repeated for each domain that you want SSL certificates for.
Steps 2, 5-7 should be repeated whenever you want to get or renew certificate.

1. Install ACMESharp PowerShell module

Install-Module -Name ACMESharp -AllowClobber

2. Import ACMESharp PowerShell module

Import-Module ACMESharp

 

3. Initial (one-time) setup

Initialize-ACMEVault

New-ACMERegistration -Contacts mailto:zhixian@hotmail.com -AcceptTos

4.  Register DNS of certificate

New-ACMEIdentifier -Dns plato.emptool.com -Alias plato_dns

5. Get challenge (to prove that you are the owner of the domain)

Complete-ACMEChallenge plato_dns -ChallengeType http-01 -Handler manual

6. Signal Let’s Encrypt to confirm your challenge answer

Submit-ACMEChallenge plato_dns -ChallengeType http-01
(Update-ACMEIdentifier plato_dns -ChallengeType http-01).Challenges | Where-Object {$_.Type -eq “http-01”}
New-ACMECertificate plato_dns -Generate -Alias plato_cert1
Submit-ACMECertificate plato_cert1
Update-ACMECertificate plato_cert1

7. Download certificates

NGINX

Get-ACMECertificate plato_cert1 -ExportCertificatePEM “C:\src\certs\plato_cert1.crt.pem”
Get-ACMECertificate plato_cert1 -ExportIssuerPEM “C:\src\certs\plato_cert1-issuer.crt.pem”

Add-Content -Value (Get-Content plato_cert1.crt.pem) -Path nginx.plato.emptool.com.pem
Add-Content -Value (Get-Content plato_cert1-issuer.crt.pem) -Path nginx.plato.emptool.com.pem

HAPROXY

ZX: Generating SSL certificates for HAPROXY is similar to NGINX, except it includes a key.

Get-ACMECertificate plato_cert1 -ExportKeyPEM “C:\src\certs\plato_cert1.key.pem”
Get-ACMECertificate plato_cert1 -ExportCertificatePEM “C:\src\certs\plato_cert1.crt.pem”
Get-ACMECertificate plato_cert1 -ExportIssuerPEM “C:\src\certs\plato_cert1-issuer.crt.pem”

Add-Content -Value (Get-Content plato_cert1.crt.pem) -Path haproxy.plato.emptool.com.pem
Add-Content -Value (Get-Content plato_cert1-issuer.crt.pem) -Path haproxy.plato.emptool.com.pem
Add-Content -Value (Get-Content plato_cert1.key.pem) -Path haproxy.plato.emptool.com.pem

 

IIS

Get-ACMECertificate plato_cert1 -ExportPkcs12 “C:\src\certs\iis.plato_cert1.pfx”

 

Advertisements

2017-07-09

How to deploy files to Windows using SFTP via Gitlab pipelines

Summary

This blog post describes how you would deploy files to a Windows Server via SFTP using Gitlab pipelines using shared runners.

The practical uptake for this is that you can deploy files for your website to be served by Internet Information Services (IIS) server using Gitlab pipelines.

Note: The context of this post is about deploying websites but the steps described can be used for deploying any type of file using Gitlab pipelines.

Contents

  1. Assumptions
  2. What are Gitlab pipelines
  3. How Gitlab pipelines work
  4. Sample .gitlab-ci.yml

Assumptions

  1. You have an working Gitlab account.
  2. You have a working Gitlab repository.
  3. You have a Windows Server
  4. You have a SFTP server running on your Windows Server and you have a working SFTP account for that server.

If you do not have a SFTP server, you can consider SFTP/SCP Server from SolarWinds.
Its not a fantastic product but it would have to do (considering that it is a free product)
The software is available at the following url after registration:
http://www.solarwinds.com/free-tools/free-sftp-server/registration

What are Gitlab pipelines

To put it simply, pipelines is Gitlab’s mechanism to perform tasks specified by you when you check-in files into your Gitlab repository. These tasks are executed by processes (dubbed "runners" in Gitlab terminology).

The runners can be grouped in shared and private (non-shared) runners.

Shared runners are hosted by Gitlab to be used by all users of Gitlab that wishes to use them). They are free to use but are limited to 2000 CI minutes per month unless you upgrade your Gitlab plan.

In comparison, private runners are setup using your own resources. After you setup your private runner, you have to register it to Gitlab in order to have Gitlab to use it.

How Gitlab pipelines work

When you check in files into your Gitlab repository, Gitlab will check for the existence of a file called ".gitlab-cl.yml". This file must be named exactly as typed (it is case-sensitive). The existence of this file tells Gitlab that there are tasks to be done. This file will list out the "jobs" for Gitlab to carry out.

Side note: As can be guessed from the file extension ".yml", this is a YAML (YAML Ain’t Markup Language) file. For details for the syntax of YAML, see http://www.yaml.org/

Sample .gitlab-ci.yml

As mentioned in the summary of this blog post, we want to setup a Gitlab pipeline that deploy to our SFTP server whenever we checked in a file. As such the below is the ".gitlab-ci.yml" file that would allow us to do that.

image: alpine

before_script:
– apk update
– apk add openssh sshpass lftp

deploy_pages:
stage: deploy
script:
– ls -al
– mkdir .public
– cp -r * .public
– echo "pwd" | sshpass -p $SFTP_PASSWORD sftp -o StrictHostKeyChecking=no zhixian@servername.somedomain.com
– lftp -e "mirror -R .public/ /test" -u zhixian,$SFTP_PASSWORD sftp://servername.somedomain.com
artifacts:
paths:
– .public
only:
– master

The following is what what each of lines do:

Line 1: Declare that "jobs" will be executed in a Docker container that use the image "alpine". The "alpine" image used here is one of the lightest Linux container, Alpine Linux. You can use other images as long as that image is in Docker store.

Line 3: The "before_script" section. Declare the actions to be carried before any jobs are executed in this section.

Line 4: Update the Alpine Linux software package manager, "apk". By default, "apk" is empty. So we need to populate it with the software catalog.

Line 5: Install the "openssh", "sshpass" and "lftp" software packages.

Line 7: Our declaration of a job call "deploy_pages"

Line 8: Indicate that this job is only to be execute in the "deploy" stage.

Quick concept of "stage": Basically, a job are executed in different stages in the order of "build", "test", and "deploy". Jobs in the same stage are executed concurrently (assuming there are sufficient runners to execute the jobs).

Line 9: The "script" section. Actions to be carried for the job are specify under here.

Line 10: List files in the docker container entry point. By default, Gitlab will dump a copy of your code repository at the container entry point. I like to see a list of the files. This is otherwise a frivolous step that is not need.

Lines 11 and 12: Make a directory call ".public" (note the period in front of "public") and copy all files at the entry point into this directory.

ZX: This step is for facilitating lftp at step 14. The problem is that Gitlab will dump a copy of the git repository at the entry point as well. But we don’t want to accidentally deploy the git repository, hence the copying of files to a sub-directory.

Line 13: Start a SFTP session to "servername.somedomain.com" using the account name "zhixian" using password stored in secret variable "$SFTP_PASSWORD".
Execute a SFTP command "pwd" and terminate the SFTP session.

ZX: This step seems frivolous, but is essential to the success of this job.
As mentioned, jobs are executed in a Docker container environment.
Hence, if we initiate any form of connection to a new SSH-based environment, system will prompt us to accept the "fingerprint-key" for that new SSH-based environment.
This line creates SFTP connection and accepts "fingerprint-key" for the SSH-based environment without prompts.

ZX: Note the "$SFTP_PASSWORD". This is a secret variable set under your Gitlab repository "Settings" section, under "Pipelines" subsection.

2017-07-09_001326

If you scroll down, you will see a "Secret variables" section like the below. The password to the SFTP account is specified here.

2017-07-09_001418

Line 14: Executes the "lftp" command. Here, we use the "mirror" feature of lftp. This feature makes a replica of the file structure of the source to the destination.

ZX: Note the "sftp://" prefix in front of the server domain name ("servername.somedomain.com"). It is important to include this to establish SFTP connectivity. If this is not specified, lftp will assume normal FTP.

Line 15: Specify the "artifacts" section. Items listed under the "artifacts" section will be available for download after the job is completed.

Line 16: Specify the "paths" section for the artifacts.

Line 17: Specify that ".public" folder is to be treated as a an artifact made available for download.

Line 18: Specify the branch of code that will cause this job would be executed.

Line 19: Specify the this job is to be executed only when someone checked-in to the "master" branch.

That’s basically all that is needed to get Gitlab to send files to your SFTP server.

References

Configuration of your jobs with .gitlab-ci.yml (https://docs.gitlab.com/ee/ci/yaml/)

2015-10-23

Setup Ubuntu (Trusty Tahr) for development

Filed under: development, ubuntu — Tags: , , , , — Zhixian @ 11:18:10 am

This blog post is on my setting up another Ubuntu Server VM.
This time I want a VM that has the common software development applications pre-installed.
As such most of the steps are similar to what I did in a previous blog post.
So on this blog post, I will start from the screen where I can select pre-packaged software.

image

Although, I was not really sure I really need the DNS server and print server, I thought “Ah well. Might as well.”
So they are included.

Setup MySql

First thing to setup is to assign a password for MySql server “root” account.
Note: You may have notice that background color have changed from purple to blue.
This was because I typed the wrong confirmation password when setting the password.

image

 

image

Setup E-mail

 

image

 

image

 

image

 

image

2015-02-09

done function not called in jQuery AJAX call

Filed under: development, web application development — Tags: , , — Zhixian @ 23:40:02 pm

In jQuery, you can write an AJAX call that is chained to a done function.
Sometimes this function does not get called.

One of things to examine is the response that you are receiving from the server.
If the response is malformed, the done function will not get triggered.

An example of a malformed response:

{ "result" : OK }

Correct reponse (note the presence of quotation marks around OK:

{ "result" : "OK" }

2014-12-30

How to install Mono on Ubuntu (Part 2 of 2)

Filed under: development — Tags: , , , , , — Zhixian @ 00:06:12 am

This is continuation of my previous blog post How to install Mono on Ubuntu.
This part focus on testing the installation to make sure that you have the minimal to compile C# source code and to run ASP.NET web applications.

Assumptions

  1. You have installed Apache HTTP Server
  2. You have followed the installation instructions in the previous blog post.
  3. You know how to create folder and files in Ubuntu.
  4. You know how to start a terminal session and run commands in it.
  5. You know how to navigate to other directories using terminal session.

Summary Steps

  1. Compiling and Running helloworld
  2. Create a Apache virtual host

Compiling and Running helloworld

Create a folder to hold your source code. For example, I use “dotnet”.

Projects_134

Inside this folder, create a file call helloworld.cs.

dotnet_135

Edit the contents of this file and replace it with the following:

// Hello1.cs
public class Hello1
{
   public static void Main()
   {
      System.Console.WriteLine("Hello, World!");
   }
}

After saving the file, start a new terminal session and navigate to the directory where you saved the file.

zhixian@SARA: ~-Projects-dotnet_136

The command to run the compiler for C# in Mono is mcs. Compile helloworld.cs by entering the following at the command-line:

mcs helloworld.cs

After running this command, you should see that you have a file call helloworld.exe in your project directory.

zhixian@SARA: ~-Projects-dotnet_137

To test run your application, enter the following command at the command-line.

./helloworld.exe

zhixian@SARA: ~-Projects-dotnet_138

You should see the words “Hello, World!” after you run the command.

Create a Apache virtual host

Now that we are sure the compiler for C# works, we will test if mod_mono can run ASP.NET applications.
What we are going to do at this section is to create a virtual host on Apache to host our ASP.NET application.

Steps

  1. Define ASP.NET application
  2. Configure Apache virtual host

Define ASP.NET application

We are going to create our test application in this section.

Create a folder to hold the contents of your ASP.NET web application.
In my case, I choose to put my file in a directory call zxtech.

Websites_139

Inside this folder, create a file call test.aspx.

zxtech_141

Edit test.aspx and replace the contents with the following:

<%@ Page Language="C#" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
  <head>
    <title>ASP Test Page</title>
  </head>
  <body>
        <form id="form1" runat="server">
          <asp:label id="lbl1" runat="server">ASP Test Page</asp:label>
        </form>
  </body>
</html>

test.aspx (~-Yandex.Disk-Websites-zxtech) - gedit_142

Save the file.
After saving the file, note the full path to the folder (press <CTRL>-L or press the button next to the Location buttons.)

You will need this information when configuring Apache virtual host.

zxtech_140

Configure Apache virtual host

Ok. Here’s where things starts to get a little sticky.
At your terminal session, navigate to Apache HTTP server’s sites directory (its /etc/apache2/sites-available/ by default) by entering the following command at the command-line:

cd /etc/apache2/sites-available/

zhixian@SARA: -etc-apache2-sites-available_143

By default, there should be 2 files in the folder, 000-default.conf and default-ssl.conf.
Make a copy of 000-default.conf for your virtual host (I am picking dev.zxtech.web as my virtual host name) by running the following command:

sudo cp ./000-default.conf ./dev.zxtech.web.conf

zhixian@SARA: -etc-apache2-sites-available_144

After the file is copied, edit the file.
Because the file is in folder owned by the superuser, you need to run the following command to edit the file:

gksudo gedit ./dev.zxtech.web.conf &

zhixian@SARA: -etc-apache2-sites-available_145

After you opened the file, you should see the following:

dev.zxtech.web.conf (-etc-apache2-sites-available) - gedit (as superuser)_146

Things to edit on this file:

  1. ServerName
  2. ServerAdmin (optional)
  3. DocumentRoot (optional)

ServerName refers to the your virtual host name that you will be using. In the below screen shot, this will be dev.zxtech.web.

ServerAdmin refers to an e-mail address that will be used when people needs to contact the server administator.

DocumentRoot refers to the folder where you store your ASP.NET application files. This will be the location of the folder that that you have noted in the previous section. In the below screen shot, this will be /home/zhixian/Yandex.Disk/Websites/zxtech.

After update, the file should look something like the below.

-dev.zxtech.web.conf (-etc-apache2-sites-available) - gedit (as superuser)_147

Things to add on this file:

  1. DirectoryIndex
  2. MonoAutoApplication
  3. AddHandler
  4. MonoServerPath
  5. MonoDebug
  6. MonoSetEnv
  7. MonoApplications
  8. Apache directory configuration
  9. Apache location configuration

DirectoryIndex refers to the default documents that will be use if user navigate to the folder without specifying a specific filename.

MonoAutoApplication refers to using a scheme in mod_mono that allows accounts on a machine to deploy ASP.NET pages without requiring to do any configuration. I prefer to do manual configuration, and so am disabling it here.

AddHandler designates files with specified file extensions to be process by mod_mono.

MonoServerPath allows you to specify the version of Mono runtime to use.
In the below screen shot, we are using the latest mono-server 4 which is equivalent to using ASP.NET 4.0.

MonoSetEnv is use to set an environment variable for use with the Mono run time used to execute the web application.
In the screen shot below, I defined MONO_IOMAP.
MONO_IOMAP is defined to take care of case-sensitivity issues. You can read more about it here.

MonoApplications is use to tell mod_mono that the ASP.NET application exists at the root directory of the web site (/) and this root directory maps (:) to the folder /home/zhixian/Yandex.Disk/Websites/zxtech (hence “/:/home/zhixian/Yandex.Disk/Websites/zxtech”)

Aside from the above Mono configuration, you may need to add an Apache directory configuration section especially if you are are hosting the web application files outside of Apache’s default DocumentRoot location which is /var/www/

 <Directory /home/zhixian/Yandex.Disk/Websites/zxtech>
 Options Indexes FollowSymLinks
 AllowOverride None
 Require all granted
 </Directory>

You will also need to add an Apache location configuration section.

 <Location "/">
 Order allow,deny
 Allow from all
 MonoSetServerAlias dev.zxtech.web
 SetHandler mono
 SetOutputFilter DEFLATE
 SetEnvIfNoCase Request_URI "\.(?:gif|jpe?g|png)$" no-gzip dont-vary
 </Location>
 <IfModule mod_deflate.c>
 AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript
 </IfModule>

After you added the sections, your configuration file should look like the below.

dev.zxtech.web.conf (-etc-apache2-sites-available) - gedit (as superuser)_153

After defining the configuration file, you can enable the website.

sudo a2ensite dev.zxtech.web.conf

zhixian@SARA: -etc-apache2-sites-available_151

The last step to get our virtual name hosting working is to edit our hosts located at /etc/hosts

zhixian@SARA: ~_150

In your text editor, add a line

127.0.1.1 dev.zxtech.web

hosts (-etc) - gedit (as superuser)_154

After this is done, you can navigate to http://dev.zxtech.web/test.aspx to test your sample web application.
You should see:

ASP Test Page - Firefox Developer Edition_155

Reference

A nice tutorial to C# can be found here (http://msdn.microsoft.com/en-us/library/aa288463%28v=vs.71%29.aspx)

2014-12-04

Google App Engine development on Ubuntu for Python (Part 1 of 2)

Filed under: development — Tags: , , , , , , , , , , , — Zhixian @ 12:21:12 pm

This blog post describes how to run Google App Engine (GAE for short) on Ubuntu.
This is a 2-part blog post.

Parts

  1. Setting up Google App Engine SDK (this article)
  2. Running Google App Engine

An aside related article, How to Add Google App Engine project to Google Developers Console.

Part 1 Summary

  1. Introduction
  2. Assumptions
  3. Python Version
  4. Installing Google App Engine

1. Introduction

Google App Engine is a PaaS (Platform as a Service) provided by Google.
Currently, GAE supports 4 programming languages.
I will be using Python as programming language of choice for this blog post.

2. Assumptions

In following this blog post, I assume:

  1. you have a working Google account
  2. you know how to start a terminal session and enter commands into it.
  3. you know how to use a browser to download files
  4. you know how to extract files from archives

Note: It may be obvious, but any time you see my name “zhixian” in any of screen shots or commands, you can safely assume that its should be referring to your username in your context.

3. Python Version

By default, Python is installed on Ubuntu.

The version of Python that I am using for this blog post is 2.7.6.
To check which version of Python you are running, typed the following command in a terminal session:

python –version

zhixian@SARA: ~_022

4. Installing Google App Engine

2 steps:

  1. Getting Google App Engine SDK (Software Development Kit)
  2. Extracting files from Google App Engine SDK
  3. (Optional) Move extracted files out of Downloads folder
  4. (Optional) Add location of extracted files to PATH environment variable

4.1 Getting Google App Engine SDK (Software Development Kit)

You can download the Google App Engine SDK at https://cloud.google.com/appengine/downloads

Steps:

  1. Enter the url https://cloud.google.com/appengine/downloads in your browser navigation bar.
  2. On the download page, click on the button “Google App Engine SDK for Python” to display the available download options
  3. Pick the link “google_appengine_1.9.15.zip” under package column for “Linux/Other Platforms” row to download the zip file.

Download the Google App Engine SDK - Google App Engine — Google Cloud Platform - Mozilla Firefox_023

4.2 Extracting files from Google App Engine SDK

After you have download the zip file, you should see the zip file in your download folder.

Downloads_025

Right-click on the zip file to display it’s context menu.
From the context menu, select “Extract Here”.

Downloads_026

After you extracted the files, you should see:

Downloads_027

I will refer to the location of google_appengine folder as GAE home directory for the rest of this blog post.

4.3 (Optional) Move extracted files out of Downloads folder

This step is optional and is entirely opinionated. It may not be appropriate for enterprise deployment.

The files that we extracted are in the default “Download” folder.
This folder may get cluttered over time.
My preference is to create a “Apps” directory and move the extracted files into this folder.
So, my final setup is something like the below:

Apps_028

google_appengine_030

4.4 (Optional) Add location of extracted files to PATH environment variable

This step is intended to provide convenience. It is not essential.

The location of the extracted files is dependent on whether you performed the previous step.

If you performed the previous step and moved the folder to an Apps folder, the location should be:
~/Apps/google_appengine

If you did not do the step, the location should be:
~/Downloads/google_appengine

As such, type the following:

export PATH=$PATH:<location_of_google_appengine>

Before you enter the following command, to examine the contents of your current PATH environment variable:

echo $PATH

zhixian@SARA: ~_032

After you enter the export PATH command, you should see a result like the below:

zhixian@SARA: ~_033

Note: Performing this command only affects the PATH environment variable for this session.
This means that every time you start a new terminal session, you have to do the same thing.
You may want to consider putting this in your .bashrc file.

I’m kind of lazy to explain what is the .bashrc file. So I’m just going to paste the screenshots of the steps for editing the .bashrc file with annotations:

At the command-line type the following command to edit the file:

pico ~/.bashrc

zhixian@SARA: ~_044

Note: pico is an alias for nano, a text-editor.

zhixian@SARA: ~_045

After you have opened the file for editing, scroll to the end of the file using the down arrow or page down key:

zhixian@SARA: ~_046

At the end of the file, enter the export PATH command.
Then press the <CTRL>-X key combination on your keyboard to exit the editor.

zhixian@SARA: ~_047

Because we did not save changes when we exit the editor, there will be a prompt asking if you want to save changes.
Press the Y key on your keyboard to save changes.

zhixian@SARA: ~_048

After you enter Y, there will be another prompt for the file to save the changes.
Press the Enter key on your keyboard to accept the default (the name of the file you are editing).

zhixian@SARA: ~_049

After you hit the Enter key, you should be back at the command-line.

To test the changes are effective, start another terminal session.
At the command-line, enter dev_a followed by the Tab key on your keyboard:

dev_a<tab>

The terminal session should automatically auto-complete your input into dev_appserver.py.

zhixian@SARA: ~_051

I will continue on how you get your first application running on Google App Engine in my next blog-post.

Blog at WordPress.com.