Thoughts of a software developer: 2015

lauantai 19. joulukuuta 2015

Using socket.io with a Go-backend

Socket.io is a library/framework which enables bidirectional communication between browser and server. The goal in mind is to update the browser status from the server side. I'll go through a simple solution I made. All the code is not here, you have to see it in github, see links at the end of the post.

Sample: https://www.uutispuro.fi/fi

Go-backend

Backend server serves all static assets, rendered html and also the server side socket.io traffic.

Template

First the Go backend is done with basic html template with speed in mind. We’re showing simple RSS data with minor tweeks. Nothing special here.

<body>
    <div id="news-container">
       {{ range .news }}
         <div class="item">
            <div class="date">{{ .PubDate.Local.Format "02.01. 15:04" }}</div>
            <div class="source">{{ .RssSource }}</div>
            <div class="category">{{ .Category.CategoryName }}</div>
            <div class="link">
                <a target="_blank" id="{{ .Id.Hex }}" href="{{ .RssLink }}">{{ .RssTitle }}</a>
            </div>
         </div>
       {{end}}
   </div>
   <script src="/public/js/socket.io.js" type="application/javascript"></script>
   <script src="/public/js/moment.min.js" type="application/javascript"></script>
   <script src="/public/js/uutispuro.js" type="application/javascript"></script>
</body>

Handling socket.io traffic

We're using a ready library which supports the latest socket.io version.

import ”github.com/googollee/go-socket.io"

Initialize server

server, err := socketio.NewServer(nil)
if err != nil {
    log.Fatal(err)
}

Listen for browsers to connect.

server.On("connection", func(so socketio.Socket) {
    so.Join("news")
    for _ = range time.Tick(10 * time.Second) {
        // get RSS titles from db
        news, _ := json.Marshal(app.Sessions.FetchRssItems("fi"))
        // broadcast 
        so.BroadcastTo("news", "message", string(news))
    }
    so.On("disconnection", func() {
        log.Println("on disconnect")
    })
})
server.On("error", func(so socketio.Socket, err error) {
    log.Println("error:", err)
})

log.Fatal(http.ListenAndServe(":1300", nil))

Frontend: javascript for handling socket.io traffic

When we get a message from the server, we parse the json, make a dom node out of it and prepend the first five titles.

window.onload = function() {
    var socket = io();
    socket.on('message', function(msg) {
        var json = JSON.parse(msg);
            for (var i = 4; i >= 0; i--) {
                if (document.getElementById(json.news[i].id) === null) {
                    var item = makeNode(json, i);
                    prepend(item);
                }
        }
        json, item = null;
    });
}

var prepend = function(firstElement) {
    var parent = document.getElementById('news-container');
    parent.insertBefore(firstElement, parent.firstChild);
    parent.removeChild(parent.lastChild)
}

Project - https://github.com/jelinden/newsfeedreader

Socket.io library for golang - https://github.com/googollee/go-socket.io
Socket.io - http://socket.io

Go html templates - https://golang.org/pkg/html/template/

keskiviikko 28. lokakuuta 2015

A seed project for universal React with a Go backend

I've been interested in making an isomorphic (or universal) web application with Go backend and React frontend for a while now. Making an application from zero is quite an effort if you code an hour now and an hour later, especially if you don't have a clear vision about the working end solution.

New univeral React seed project with a Go backend

So I started with a seed project which I can use for many quick try outs. Mostly for myself but anyone can use it. You can see it running at go-react-seed.uutispuro.fi.

React side of the application is simple. Package.json handles all the necessary tasks, no excess mile long grunt or gulp files.

"scripts": {
  "build-dir": "rm -rf build && mkdir build && cp public/js/* build",
  "babelify": "babel build --out-file build/babelified.js",
  "browserify": "browserify build/babelified.js -g uglifyify --outfile build/bundle.js",
  "build": "npm run build-dir && npm run babelify && npm run browserify"
}

At the moment registration, login and logout are working. Page showing all the members is accessible only to user with an admin role. Pages are responsive although there is not much content. If there where, it wouldn't be a seed project.

Backend

Go is the language to go. Echo is used for server framework. It's easy to use, extensible and really fast. There are many middleware available and it's easy to add them yourself. We're rendering the same React javascript code on the server side as on the client side.

Database

Database used is Redis, but that is somewhat easily changeable if needed. The first one to make the registration gets admin role.

func (a *Application) createUser(c *echo.Context) error {

role := domain.Role{Name: domain.Normal}

if a.Redis.DbSize() == 0 {

role = domain.Role{Name: domain.Admin}

}

To be done

Assets versioning
Member page is not informational to others than admin user, don't show it to others
Verification of new user with email
Forgot my password functionality
Running with Raspberry pi would be nice, should be doable (http://www.mccarroll.net/blog/v8_pi2/index.html)

Related blogposts

Revisited: Isomorphic React.js with Go backend

Isomorphic React.js with Go backend

sunnuntai 18. lokakuuta 2015

Using New Relic with a Go web application

I stumbled into a great middleware named gorelic.

It's too easy to start using it and you get useful information about your app.

package main

import (
    "github.com/labstack/echo"
    "github.com/syntaqx/echo-middleware/gorelic"
)

func main() {
    e := echo.New()

    // Attach middleware
    gorelic.InitNewRelicAgent("YOUR_LICENSE_KEY", "YOUR_APPLICATION_NAME", true)
    e.Use(gorelic.Handler())

    e.Run(":8080")
}

Memory usage

For example from the image below, I straight away knew that handling the right to view data for user with errors, seems not to be the right way. On such a page the response time rises immediately (green and purple lines).

Response time

tiistai 22. syyskuuta 2015

A simple weather slackbot made with Go

Inspired by rapidloops mybot I made a slackbot which tells you the current weather when asked.
You can view the code at https://github.com/jelinden/slackbot.

The command which it listens is simple

@bot weather

You can add a bot easily your self. Open dropdown menu after channel name and choose Add a service integration.

Find and Click view on Bots.

Give the bot a username.

Get the API Token, you can fill in more information about the bot too, image and such, but it's not necessary.

perjantai 18. syyskuuta 2015

Revisited: Isomorphic React.js with Go backend

I found an interesting project at github named selfjs (https://github.com/nmerouze/selfjs). It's a small project, but it uses an other project named v8worker (github.com/ry/v8worker). The idea is great, to use chrome's very own v8 engine to interpret javascript. Selfjs even claims it's faster than with node which also is built on v8. This is of course server side rendering we're talking about.

Earlier I wrote about otto, and didn't think it was fast enough. But v8worker just might be. I made an almost equal application with selfjs and v8worker.

http://isomorphic.uutispuro.fi/ (using otto)
http://isomorphic2.uutispuro.fi/ (using v8worker)

The first page rendering takes about 0.27 seconds on my Mac. With a little caching that should be liveable.

source code
https://github.com/jelinden/go-isomorphic-react-v8

perjantai 5. kesäkuuta 2015

Sharing a directory in boot2docker

Don't know why, but it seems to be really hard to find an easy way to share a folder in boot2docker.

$ boot2docker ssh

docker@boot2docker:~$ sudo mkdir /mnt/work

docker@boot2docker:~$ sudo mount -t vboxsf -o uid=1000,gid=50 /work /mnt/work

sunnuntai 31. toukokuuta 2015

Isomorphic React.js with Go backend

Go is a great language for making web sites. Something in component based React is absolutely brilliant after getting to know bloat Angularjs. Combining Go for server side rendering and api backend, and React for client side rendering sounds like a dream come true. No need for prerendering services which is really really silly (First we make a client side rendering website and then we can't use it for everything).

The code which I'm writing here is running at http://isomorphic.uutispuro.fi/.

I had earlier come across Echo, (Echo is a fast HTTP router (zero memory allocation) and micro web framework in Go), which I had to try.

import "github.com/labstack/echo"

e := echo.New()

// server side rendered pages
e.Get("/", index)
e.Get("/anotherpage", anotherpage)

// static files
e.Static("/public/js", "public/js")
e.Static("/public/css", "public/css")

// backend api proxy
e.Get("/api/frontpage", apiFrontPage)
e.Get("/api/anotherpage", apiAnotherPage)

e.Run(":3000")

Quite simple really. Easy to add gzip and other middleware.

A little illustration of what we are doing

Server side rendering

We will get a RSS feed from BBC scheduled and in the background and save the fetched feed and it's rendered version for later use.

First the scheduling:
func tick() {

// schelude ticker for every 70 seconds
ticker := time.NewTicker(70 * time.Second)
for {

// fetch the rss feed
fetchFeed()

// render the rss feed
reactIndex()
<-ticker.C
}
}

Rendering:

func reactIndex() {
v := newRenderer([]string{"public/js/frontpage.js", "public/js/common.js"}).
runCmd(`
var data = ` + rss + `;
React.renderToString(News({'data' : data}));
`)
sValue, err := v.ToString()
if err != nil {
fmt.Println(err)
}
frontPageRendered = sValue
}

We make a newRenderer to which we give two needed javascript files and then run the runCmd which interprets the javascript into html. This is done with React ids, so that when the same is done in the client, rendering doesn't need to be done again.

Client side rendering

What we do at the client end to start up React and how do we handle going from page to an other?

We add a client.js into use for the browser.

var xmlhttp = new XMLHttpRequest();

var route = function() {
if (window.location.pathname === '/') {
xmlhttp.onreadystatechange = function() {
if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
var json = xmlhttp.responseText;
React.render(News({'data' : JSON.parse(json)}), document.getElementById('body'));
}
}
xmlhttp.open("GET", "/api/frontpage", true);
xmlhttp.send();
}

if (window.location.pathname === '/anotherpage') {
xmlhttp.onreadystatechange = function() {
if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
var json = xmlhttp.responseText;
React.render(Another({'data' : JSON.parse(json)}), document.getElementById('body'));
}
}
xmlhttp.open("GET", "/api/anotherpage", true);
xmlhttp.send();
}
}
route();

We have two api routes, one for root page and one for the second page. Both routes get json from api endpoints. After the function we call for the initial React check (React.render). The rendering would actually be enough after full page load, but this is a simplified solution.

At frontpage.js and anotherpage.js we have the navigation links (duplicate code, should be a components on its own).

React.DOM.div({className: "pure-menu pure-menu-horizontal"},
React.createElement("a", {
className: "pure-menu-heading pure-menu-link",
href: "/",
onClick: clickHandler
}, "Home"),
React.DOM.ul({className: "pure-menu-list"},
React.DOM.li({className: "pure-menu-item"},
React.createElement("a", {
href: "/anotherpage",
onClick: clickHandler
}, "Another page")
)
)
)

Clicks for these navigation links call for clickHandler which is in common.js

var clickHandler = function(e) {
e.preventDefault()
history.pushState({}, "", e.target.href);
route();
}

First we disable making the whole page load, second we make history.pushState for altering the browser location bar, and finally call for route to change the page content according to changed location.

You can view the whole code at https://github.com/jelinden/go-isomorphic-react and see it in action at http://isomorphic.uutispuro.fi/.

If someone uses this for own purposes, bare in mind that there are many unfinished things. Client side is made so that "it works". Doesn't mean that everything is polished or that everything works. For example entering backspace doesn't load the page content as it should.

Another major setback is the fact that the javascript interpreter is way too slow. You can't use it to render server side on the fly. On my Mac rendering front page takes about 4 seconds.

lauantai 9. toukokuuta 2015

Spring-boot and logging with logback

I'm making a website with spring-boot and wanted to log the remote host or IP. Couldn't do it straight away and it took a while to find that I needed a filter to achieve what I wanted.

<appender name="dailyRollingFileAppender" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <File>logs/mylog.log</File>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
        <FileNamePattern>logs/mylog.%d{yyyy-MM-dd}.log</FileNamePattern>
        <maxHistory>5</maxHistory>
    </rollingPolicy>

    <encoder>
        <Pattern>%d{HH:mm:ss.SSS} %-4r [%thread] %-0level %X{req.remoteHost} %X{req.requestURI} %logger{20} - %msg %n</Pattern>
    </encoder>
</appender>

Before setting MDCInsertingServletFilter, patterns like %X{req.*} didn't work at all.

With spring-boot, you can set the filter with:

@Bean
public MDCInsertingServletFilter mdcInsertingServletFilter() {
    return new MDCInsertingServletFilter();
}

and the patterns work nicely.

lauantai 21. helmikuuta 2015

Setting a cookie with Golang

Couldn't find a good example about how to set a cookie to response with Golang using Gin.

So, here is what I did

r.LoadHTMLTemplates("templates/*")
r.GET("/en", func(c *gin.Context)
 
{
    expire
 
:=
 time
.Now
().AddDate
(
1
,
 
0
,
 
1
)
    cookie
 :=
 http.Cookie
{"cookieName", "cookieValue",
        "/", ".domain.com", expire, expire.Format(time.UnixDate), 
        41472000, false, false, "cookieName=cookieValue", 
        []string{"cookieName=cookieValue"}}
     http.SetCookie(context.Writer, &cookie)
     c.HTML(200, "index.html", gin.H{"title": "Main website"})
})

From documentation (http://golang.org/src/net/http/cookie.go)

type Cookie struct {

Name string

Value string

Path string

Domain string

Expires time.Time

RawExpires string

// MaxAge=0 means no 'Max-Age' attribute specified.

// MaxAge<0 means delete cookie now, equivalently 'Max-Age: 0'

// MaxAge>0 means Max-Age attribute present and given in seconds

MaxAge int

Secure bool

HttpOnly bool

Raw string

Unparsed []string // Raw text of unparsed attribute-value pairs

}