Thoughts of a software developer

lauantai 5. maaliskuuta 2016

Raspberry Pi cluster with Kubernetes

I got my first Raspberry Pi as a gift when I was an a Architecting on AWS -course. I’ve been mostly just playing with it, but wanted to use it for something useful. Then I read a blog post about making a cluster out of them and got really interested.

Creating a Raspberry Pi cluster running Kubernetes, the shopping list (Part 1) and Creating a Raspberry Pi cluster running Kubernetes, the installation (Part 2).

I ordered three (so I have one less than in the original recipe) more Raspberrys and started making a cluster. Initially I was thinking about some new database cluster but then changed to scaling Uutispuro, a rss feed title lister that I’ve been making for quite some time now. It uses Mongodb from outside of the Kubernetes.

Making the actual cluster went nicely with the help of the blog posts I followed. I did also setup ntpd and used fi_FI.UTF-8 as a locale for each Pi. Each worker node connected nicely to master, only the last one had a hickup of some kind (it got stuck to “NotReady”) but restart helped.

Docker Hub

It was a surprise for me that I had to use Docker Hub for getting the docker image to Kubernetes. At least it should be possible to use the image straigth away that I'm making.

# build a docker image
docker build -t jelinden/newsfeedreader:0.2 .
# push to docker hub
docker push jelinden/newsfeedreader:0.2
# get the image from docker hub and expose port 1300 and run it in one of the Pi's
kubectl run newsfeedreader --image=docker.io/jelinden/newsfeedreader:0.2 --port=1300

You can view what's going on inside the cluster with

kubectl get events -w

And then you can scale it up to run on three instances

kubectl scale rc newsfeedreader --replicas=3

I couldn’t get Docker Hub to work with a private repo, you can read more about it at Kubernetes PullImageError using Docker Hub with a private image. Luckily for me it doesn't matter if it's public or private.

What next?

Worker node ports are only accessible to master node, so I have to add a load balancer to the setup. Adding a haproxy or nginx is too easy, so I will make my own with Go.

lauantai 2. tammikuuta 2016

Free SSL sertificates from Letsencrypt

SSL sertificates have been too hard to get up and working. Addition to that, they mostly have been a bit costly.

Now there seems to be a working alternative. Letsencrypt offers an easy solution to fetch and use sertificates.

Here's an example what I did.

1. Get the scripts

git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt

2. For getting the sertificate, you need to have either 80 or 443 port free to be used (for a moment only), this uses port 443

./letsencrypt-auto certonly --standalone -d uutispuro.fi -d www.uutispuro.fi --standalone-supported-challenges tls-sni-01

3. Add the following lines (altered to your domain of course) to your nginx conf

ssl_certificate /etc/letsencrypt/live/uutispuro.fi/cert.pem;
ssl_certificate_key /etc/letsencrypt/live/uutispuro.fi/fullchain.pem;

See https://www.uutispuro.fi/en to see it in action :)

You do need to update it periodically, every three months or so, but that is nothing else than doing the step 2. You can run it in crontab easily.

lauantai 19. joulukuuta 2015

Using socket.io with a Go-backend

Socket.io is a library/framework which enables bidirectional communication between browser and server. The goal in mind is to update the browser status from the server side. I'll go through a simple solution I made. All the code is not here, you have to see it in github, see links at the end of the post.

Sample: https://www.uutispuro.fi/fi

Go-backend

Backend server serves all static assets, rendered html and also the server side socket.io traffic.

Template

First the Go backend is done with basic html template with speed in mind. We’re showing simple RSS data with minor tweeks. Nothing special here.

<body>
    <div id="news-container">
       {{ range .news }}
         <div class="item">
            <div class="date">{{ .PubDate.Local.Format "02.01. 15:04" }}</div>
            <div class="source">{{ .RssSource }}</div>
            <div class="category">{{ .Category.CategoryName }}</div>
            <div class="link">
                <a target="_blank" id="{{ .Id.Hex }}" href="{{ .RssLink }}">{{ .RssTitle }}</a>
            </div>
         </div>
       {{end}}
   </div>
   <script src="/public/js/socket.io.js" type="application/javascript"></script>
   <script src="/public/js/moment.min.js" type="application/javascript"></script>
   <script src="/public/js/uutispuro.js" type="application/javascript"></script>
</body>

Handling socket.io traffic

We're using a ready library which supports the latest socket.io version.

import ”github.com/googollee/go-socket.io"

Initialize server

server, err := socketio.NewServer(nil)
if err != nil {
    log.Fatal(err)
}

Listen for browsers to connect.

server.On("connection", func(so socketio.Socket) {
    so.Join("news")
    for _ = range time.Tick(10 * time.Second) {
        // get RSS titles from db
        news, _ := json.Marshal(app.Sessions.FetchRssItems("fi"))
        // broadcast 
        so.BroadcastTo("news", "message", string(news))
    }
    so.On("disconnection", func() {
        log.Println("on disconnect")
    })
})
server.On("error", func(so socketio.Socket, err error) {
    log.Println("error:", err)
})

log.Fatal(http.ListenAndServe(":1300", nil))

Frontend: javascript for handling socket.io traffic

When we get a message from the server, we parse the json, make a dom node out of it and prepend the first five titles.

window.onload = function() {
    var socket = io();
    socket.on('message', function(msg) {
        var json = JSON.parse(msg);
            for (var i = 4; i >= 0; i--) {
                if (document.getElementById(json.news[i].id) === null) {
                    var item = makeNode(json, i);
                    prepend(item);
                }
        }
        json, item = null;
    });
}

var prepend = function(firstElement) {
    var parent = document.getElementById('news-container');
    parent.insertBefore(firstElement, parent.firstChild);
    parent.removeChild(parent.lastChild)
}

Project - https://github.com/jelinden/newsfeedreader

Socket.io library for golang - https://github.com/googollee/go-socket.io
Socket.io - http://socket.io

Go html templates - https://golang.org/pkg/html/template/

keskiviikko 28. lokakuuta 2015

A seed project for universal React with a Go backend

I've been interested in making an isomorphic (or universal) web application with Go backend and React frontend for a while now. Making an application from zero is quite an effort if you code an hour now and an hour later, especially if you don't have a clear vision about the working end solution.

New univeral React seed project with a Go backend

So I started with a seed project which I can use for many quick try outs. Mostly for myself but anyone can use it. You can see it running at go-react-seed.uutispuro.fi.

React side of the application is simple. Package.json handles all the necessary tasks, no excess mile long grunt or gulp files.

"scripts": {
  "build-dir": "rm -rf build && mkdir build && cp public/js/* build",
  "babelify": "babel build --out-file build/babelified.js",
  "browserify": "browserify build/babelified.js -g uglifyify --outfile build/bundle.js",
  "build": "npm run build-dir && npm run babelify && npm run browserify"
}

At the moment registration, login and logout are working. Page showing all the members is accessible only to user with an admin role. Pages are responsive although there is not much content. If there where, it wouldn't be a seed project.

Backend

Go is the language to go. Echo is used for server framework. It's easy to use, extensible and really fast. There are many middleware available and it's easy to add them yourself. We're rendering the same React javascript code on the server side as on the client side.

Database

Database used is Redis, but that is somewhat easily changeable if needed. The first one to make the registration gets admin role.

func (a *Application) createUser(c *echo.Context) error {

role := domain.Role{Name: domain.Normal}

if a.Redis.DbSize() == 0 {

role = domain.Role{Name: domain.Admin}

}

To be done

Assets versioning
Member page is not informational to others than admin user, don't show it to others
Verification of new user with email
Forgot my password functionality
Running with Raspberry pi would be nice, should be doable (http://www.mccarroll.net/blog/v8_pi2/index.html)

Related blogposts

Revisited: Isomorphic React.js with Go backend

Isomorphic React.js with Go backend

sunnuntai 18. lokakuuta 2015

Using New Relic with a Go web application

I stumbled into a great middleware named gorelic.

It's too easy to start using it and you get useful information about your app.

package main

import (
    "github.com/labstack/echo"
    "github.com/syntaqx/echo-middleware/gorelic"
)

func main() {
    e := echo.New()

    // Attach middleware
    gorelic.InitNewRelicAgent("YOUR_LICENSE_KEY", "YOUR_APPLICATION_NAME", true)
    e.Use(gorelic.Handler())

    e.Run(":8080")
}

Memory usage

For example from the image below, I straight away knew that handling the right to view data for user with errors, seems not to be the right way. On such a page the response time rises immediately (green and purple lines).

Response time

tiistai 22. syyskuuta 2015

A simple weather slackbot made with Go

Inspired by rapidloops mybot I made a slackbot which tells you the current weather when asked.
You can view the code at https://github.com/jelinden/slackbot.

The command which it listens is simple

@bot weather

You can add a bot easily your self. Open dropdown menu after channel name and choose Add a service integration.

Find and Click view on Bots.

Give the bot a username.

Get the API Token, you can fill in more information about the bot too, image and such, but it's not necessary.

perjantai 18. syyskuuta 2015

Revisited: Isomorphic React.js with Go backend

I found an interesting project at github named selfjs (https://github.com/nmerouze/selfjs). It's a small project, but it uses an other project named v8worker (github.com/ry/v8worker). The idea is great, to use chrome's very own v8 engine to interpret javascript. Selfjs even claims it's faster than with node which also is built on v8. This is of course server side rendering we're talking about.

Earlier I wrote about otto, and didn't think it was fast enough. But v8worker just might be. I made an almost equal application with selfjs and v8worker.

http://isomorphic.uutispuro.fi/ (using otto)
http://isomorphic2.uutispuro.fi/ (using v8worker)

The first page rendering takes about 0.27 seconds on my Mac. With a little caching that should be liveable.

source code
https://github.com/jelinden/go-isomorphic-react-v8