Näytetään tekstit, joissa on tunniste golang. Näytä kaikki tekstit
Näytetään tekstit, joissa on tunniste golang. Näytä kaikki tekstit

perjantai 2. kesäkuuta 2017

Trying out Graphql with Golang and Vue.js

I've made normal Rest APIs for many years now. For some time now, there has been interest for Facebooks GraphQL especially on the front-end side. I too have been curious about it but haven't had the change to play with it. So, I made a small project to test it and to test Vue.js too which also is new for me. I have used Angular a bit and more React but Vue.js was something totally new.

GraphQL with Go

There actually is a ready project for using GraphQL so it's quite simple to start using it on the backend side.

I couldn't think of any really good topic to make so I wrote some Star Wars characters into a file and used those.

var personType = graphql.NewObject(
  graphql.ObjectConfig{
    Name:   "person",
    Fields: fields,
  },
)

There is only one type and it's definition is simple enough.

var queryType = graphql.NewObject(graphql.ObjectConfig{
  Name: "RootQuery",
  Fields: graphql.Fields{
    "person": &graphql.Field{
      Type: personType,
      Args: args,
      Resolve: func(p graphql.ResolveParams) (interface{}, error) {
        return filterPerson(data, p.Args), nil
      },
    },
    "personList": &graphql.Field{
      Type: graphql.NewList(personType),
      Resolve: func(p graphql.ResolveParams) (interface{}, error) {
        return data, nil
      },
    },
  },
})

Now, the query type includes two fields to make queries. Person field filters the data with a query parameter from the request. PersonList field just returns the whole data or list of Star Wars characters.

schema, _ = graphql.NewSchema(
  graphql.SchemaConfig{
    Query: queryType,
  },
)

Schema defines the capabilities of the server with queryType.

func main() {
  err := importJSONDataFromFile(jsonDataFile)
  if err != nil {
    fmt.Printf("Error: %s\n", err.Error())
    return
  }

  http.HandleFunc("/graphql", func(w http.ResponseWriter, r *http.Request) {
    result := executeQuery(r.URL.Query()["query"][0], schema)
    w.Header().Add("Access-Control-Allow-Origin", "*")
    json.NewEncoder(w).Encode(result)
  })

  fs := http.FileServer(http.Dir("frontend/dist"))
  http.Handle("/", fs)

  fmt.Println("Now server is running on port 8000")
  http.ListenAndServe(":8000", nil)
}

The main function of the server does four things. First, it gets the Star Wars character json and makes the personType "fields" and queryType "data" available. Second, it sets listening endpoint to /graphql for queries. Third, it defines a file server for static content for root "/" endpoint. And for last, it starts the server.

At this point, qraphql queries already work. If your start the server, and query it for example with:

curl http://localhost:8000/graphql?query={person(name:%22Darth%22){id,name,surname}}

You get an answer:

{
  data: {
    person: {
      id: "1",
      name: "Darth",
      surname: "Vader"
    }
  }
}

Vue.js

To really test what the difference is between using ordinary Rest APIs and GraphQL I set up a small front-end to fetch and show the data.


I first set up the vue.js project under front-end directory, in production it would be delivered from there. In development it can be run separetily with hot reloading and the works.

cd frontend
npm install -g vue-cli
vue init webpack graphql-vue-example
mv graphql-vue-example/* .
rmdir graphql-vue-example
npm install
npm run dev

Now, this sets up an initial project, it needs some work to get all things like we want them. Almost everything that needs to be changed, is at "frontend/src/components/Persons.vue". It contains the html template, vue javascript for the page and scoped styles.

<template>
  <div class="personclass">
    <h1>{{ msg }}</h1>
    <input id="personName" type="text" value="Darth" v-model="personName"></input>
    <button class="btn btn-primary" v-on:click="getPerson()">Get Star wars person</button>
    <h2>One fetched person</h2>
    <div>{{ person.person.id }} {{ person.person.name }} {{ person.person.surname }}</div>
    <h2>List of all</h2>
    <div v-for="value in persons.personList">
      {{ value.id }} {{ value.name }} {{ value.surname }}
    </div>
  </div>
</template>

If you write a name of a character to the input field and then click the button, it goes and fetches the character in question. Of course, only if there is a match. There also is the whole list, which is fetched immediately when the page is rendered.

<script>
export default {
  name: 'personclass',
  data () {
    return {
      msg: 'Star Wars persons',
      person: {person: {id: '', name: '', surname: ''}},
      persons: ''
    }
  },
  mounted () {
    this.getPersons()
  },
  methods: {
    getPerson () {
      console.log(this.personName)
      this.$http.get('http://localhost:8000/graphql?query=' + 
          encodeURIComponent('{person(name:"' + this.personName + '"){id,name,surname}}'))
          .then(response => {
        this.person = JSON.parse(response.body).data
      }, response => {
        console.log(response)
      })
    },
    getPersons () {
      this.$http.get('http://localhost:8000/graphql?query=' + 
          encodeURIComponent('{personList{id,name,surname}}')).then(response => {
        this.persons = JSON.parse(response.body).data
      }, response => {
        console.log(response)
      })
    }
  }
}
</script>

The Vue javascript part is simple enough. There are two methods, getPersons gets all the Star Wars characters and it's called when the page is mounted. Method getPerson is called from the html page when the button is clicked and it returns only the character which is being searched.


The whole example project in github: https://github.com/jelinden/graphql-vue-example

References

https://facebook.github.io/react/blog/2015/05/01/graphql-introduction.html
https://github.com/graphql-go/graphql
https://vuejs.org/


sunnuntai 19. kesäkuuta 2016

Cache on the background with golang

Caching based on requests

Usually caches on the web are made so that the cache is filled with requests coming in from the browser. Positive about this is that you cache the most used requests and not the ones which are not used. The downside is that in case of errors or timeouts, you really have to think what to do. And what about expiration and the size of the cache?

Caching on the background

If you know what to fetch and you know the requests what to get before hand, you can easily schedule the cache fetching. Although I'm not using a popular cache like LRU here, it's certainly possible.

A simple example

This is a simple example for a cache. You can get, add and remove items from the cache.
package main

import (
    "github.com/streamrail/concurrent-map"
)

var cache cmap.ConcurrentMap

func init() {
    cache = cmap.New()
}

func AddItemToCache(key string, value string) {
    cache.Set(key, value)
}

func GetItemFromCache(key string) string {
    if cache.Has(key) {
        if tmp, ok := cache.Get(key); ok {
            return tmp.(string)
        }
    }
    return ""
}

func removeItem(key string) {
    cache.Remove(key)
}

Let’s get weather information on the background.
func fetchWeather() *Weather {
    var w Weather
    resp, err := client.Get(weatherURL)
    if err != nil {
        fmt.Println(err)
        return nil
    }
    defer resp.Body.Close()
    buf, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        fmt.Println(err)
        return nil
    }
    if err = json.Unmarshal(buf, &w); err != nil {
        fmt.Println(err)
    }
    return &w
}

Marshal it to json string
func weatherToCache() {
    fmt.Println("fetching weather")
    if w := fetchWeather(); w != nil {
        j, err := json.Marshal(&w.Query.Results.Channel.Item)
        if err != nil {
            fmt.Println(err)
        } else {
            AddItemToCache(weatherURL, string(j))
        }
    }
}

and schedule fetching it
func weatherSchedule() {
    for range time.Tick(time.Hour * 1) {
        weatherToCache()
    }
}

In main function We use iris to get the cached weather information after we put the weather scheduling to run on the background.
func main() {
    weatherToCache()
    go weatherSchedule()
    iris.Get("/weather", func(c *iris.Context) {
        c.Text(200, GetItemFromCache(weatherURL))
    })
    iris.Listen(":8080")
}

Used or mentioned dependencies
iris-go.com
Concurrent map
LRU cache

See the whole project at https://github.com/jelinden/go-background-cache-example

lauantai 19. maaliskuuta 2016

Saving page loads to InfluxDB through NATS and showing it as a graph with c3.js

I wanted to try out updating graphs in real time, and at the same time try something new. I’ve heard good things about Nats and InfluxDB so these were my preferred choises. I already have a web site www.uutispuro.fi, which I'm also using.

Stack used

  • c3.js - made on top of d3.js, a javascript library for graphs
  • Nats - A fast message broker
  • InfluxDB - Time series database

Installing

First installing Nats and InfluxDB were really easy. Download Nats binary and run it. For InfluxDB there are apt-get installation instructions at https://influxdata.com/downloads.

The code

To get the data to the database, I first needed to send the messages for each page load to Nats. The client is easy to use. For data to be saved I chose the ip address of the server so that I can make own graphs for each of them in the future. URI is the path which was asked, this can be used to show the pages which are read the most. User agent is also an interesting field of information. Time is used for the time based query from the database.

uri := "\"uri\":\"" + c.Request().URI() + "\""
userAgent := "\"userAgent\":\"" + c.Request().Header().Get("user-agent") + "\""
localIP := "\"localIP\":\"" + nats.LocalIP + "\""
time := "\"time\":\"" + time.Now().UTC().String() + "\""
nats.NatsConn.Publish("click", []byte("{"+time+", "+uri+", "+userAgent+", "+localIP+"}"))
if err := next.Handle(c); err != nil {
  c.Error(err)
}

And then in plotter project, I need to listen to Nats.

func GetClicks() {
  NewInflux()
  nc, err := nats.Connect("nats://192.168.0.5:4222", 
    nats.MaxReconnects(60), 
    nats.ReconnectWait(2*time.Second))
  if err != nil {
      fmt.Println(err)
    return
  }
  defer nc.Close()
  ch := make(chan *nats.Msg, 128)
  _, err = nc.ChanSubscribe("click", ch)
  if err != nil {
    fmt.Println(err)
    return
  }

  for {
    msg := <-ch
    var data map[string]interface{}
    if err := json.Unmarshal(msg.Data, &data); err != nil {
      panic(err)
    }
    Save(data)
  }
}

And save the data to the database.

func Save(click map[string]interface{}) {
  bp, err := client.NewBatchPoints(client.BatchPointsConfig{
    Database:  influx.DbName,
    Precision: "s",
  })
  if err != nil {
    log.Println(err)
  }

  stamp, _ := time.Parse("2006-01-02 15:04:05 -0700 MST", click["time"].(string))
  pt, err := client.NewPoint("click", map[string]string{}, click, stamp)
  if err != nil {
    log.Println(err)
  }
  bp.AddPoint(pt)
  err = influx.Conn.Write(bp)
  if err != nil {
    log.Println(err)
  }
}

Now to the exiting part. We have the data in the database and it's easy to fetch it from there.

func GetList() map[string]int64 {
  q := client.NewQuery("select count(localIP) from click 
      where time > now() - 1h 
      group by time(1m) fill(0)",
      "plotter", "s")
  var fields = make(map[string]int64)
  response, err := influx.Conn.Query(q)
  if err != nil || response.Error() != nil {
    fmt.Println(err)
  } else {
    for _, result := range response.Results {
      for _, value := range result.Series[0].Values {
        id := strconv.FormatInt(convertType(value[0]), 10)
        val := convertType(value[1])
        fields[id] = val
      }
    }
  }
  return fields
}

What do we do with the data? We'll send it with websockets to the browser every second.

e.Get("/ws", standard.WrapHandler(websocket.Handler(func(ws *websocket.Conn) {
  for range time.Tick(time.Second) {
    clicks, err := json.Marshal(app.GetList())
      if err != nil {
        fmt.Println(err)
      }
      err = websocket.Message.Send(ws, string(clicks))
      if err != nil {
        fmt.Printf("client closed connection %s\n", err)
        break
      }
    }
})))

In the browser we take a websocket connection and listen.

window.onload = function() {
  writeState("document loaded");
  if ('WebSocket' in window) {
    var connection = new WebSocket('ws://uutispuro.fi:7000/ws');
    connection.onopen = function() {
      writeState("connection open");
    }
    connection.onclose = function() {
      writeState("connection closed");
    }
    connection.onerror = function(error) {
      writeState("error: " + error);
    }
    connection.onmessage = function(e) {
      var msg = JSON.parse(e.data);
      writeState("got a message");
      connection.send("ack");
      arr = ['per minute'];
      Object.keys(msg).map(function(key, i) {
        if (i == 0) {
          arr.push(0);
        } else {
          arr.push(msg[key]);
        }
      });
      chart.load({
        columns: [
          arr
        ]
      });
    }
  } else {
    writeState("You're browser does not support websockets.");
  }
}

And draw the graph with the data sent from the server.

var chart = c3.generate({
  data: {
    columns: [
      ['per minute', 0],
    ],
    type: 'bar'
  },
  bar: {
    width: {
      ratio: 1.0
    }
  },
  axis: {
    x: {
      label: 'minutes',
      inverted: true
    },
    y: {
      label: 'page loads'
    }
  }
});

Links

lauantai 19. joulukuuta 2015

Using socket.io with a Go-backend

Socket.io is a library/framework which enables bidirectional communication between browser and server. The goal in mind is to update the browser status from the server side. I'll go through a simple solution I made. All the code is not here, you have to see it in github, see links at the end of the post.

Go-backend

Backend server serves all static assets, rendered html and also the server side socket.io traffic.


Template

First the Go backend is done with basic html template with speed in mind. We’re showing simple RSS data with minor tweeks. Nothing special here.

<body>
    <div id="news-container">
       {{ range .news }}
         <div class="item">
            <div class="date">{{ .PubDate.Local.Format "02.01. 15:04" }}</div>
            <div class="source">{{ .RssSource }}</div>
            <div class="category">{{ .Category.CategoryName }}</div>
            <div class="link">
                <a target="_blank" id="{{ .Id.Hex }}" href="{{ .RssLink }}">{{ .RssTitle }}</a>
            </div>
         </div>
       {{end}}
   </div>
   <script src="/public/js/socket.io.js" type="application/javascript"></script>
   <script src="/public/js/moment.min.js" type="application/javascript"></script>
   <script src="/public/js/uutispuro.js" type="application/javascript"></script>
</body> 

Handling socket.io traffic

We're using a ready library which supports the latest socket.io version.
import ”github.com/googollee/go-socket.io"

Initialize server
server, err := socketio.NewServer(nil)
if err != nil {
    log.Fatal(err)
}

Listen for browsers to connect.
server.On("connection", func(so socketio.Socket) {
    so.Join("news")
    for _ = range time.Tick(10 * time.Second) {
        // get RSS titles from db
        news, _ := json.Marshal(app.Sessions.FetchRssItems("fi"))
        // broadcast 
        so.BroadcastTo("news", "message", string(news))
    }
    so.On("disconnection", func() {
        log.Println("on disconnect")
    })
})
server.On("error", func(so socketio.Socket, err error) {
    log.Println("error:", err)
})
log.Fatal(http.ListenAndServe(":1300", nil))


Frontend: javascript for handling socket.io traffic


When we get a message from the server, we parse the json, make a dom node out of it and prepend the first five titles.

window.onload = function() {
    var socket = io();
    socket.on('message', function(msg) {
        var json = JSON.parse(msg);
            for (var i = 4; i >= 0; i--) {
                if (document.getElementById(json.news[i].id) === null) {
                    var item = makeNode(json, i);
                    prepend(item);
                }
        }
        json, item = null;
    });
}

var prepend = function(firstElement) {
    var parent = document.getElementById('news-container');
    parent.insertBefore(firstElement, parent.firstChild);
    parent.removeChild(parent.lastChild)
}
Socket.io library for golang - https://github.com/googollee/go-socket.io
Socket.io - http://socket.io

keskiviikko 28. lokakuuta 2015

A seed project for universal React with a Go backend

I've been interested in making an isomorphic (or universal) web application with Go backend and React frontend for a while now. Making an application from zero is quite an effort if you code an hour now and an hour later, especially if you don't have a clear vision about the working end solution.

New univeral React seed project with a Go backend

So I started with a seed project which I can use for many quick try outs. Mostly for myself but anyone can use it. You can see it running at go-react-seed.uutispuro.fi.

React side of the application is simple. Package.json handles all the necessary tasks, no excess mile long grunt or gulp files.
"scripts": {
  "build-dir": "rm -rf build && mkdir build && cp public/js/* build",
  "babelify": "babel build --out-file build/babelified.js",
  "browserify": "browserify build/babelified.js -g uglifyify --outfile build/bundle.js",
  "build": "npm run build-dir && npm run babelify && npm run browserify"
}


At the moment registration, login and logout are working.  Page showing all the members is accessible only to user with an admin role. Pages are responsive although there is not much content. If there where, it wouldn't be a seed project.


Backend

Go is the language to go. Echo is used for server framework. It's easy to use, extensible and really fast. There are many middleware available and it's easy to add them yourself. We're rendering the same React javascript code on the server side as on the client side.


Database

Database used is Redis, but that is somewhat easily changeable if needed. The first one to make the registration gets admin role.
func (a *Application) createUser(c *echo.Context) error {
    role := domain.Role{Name: domain.Normal}
    if a.Redis.DbSize() == 0 {
        role = domain.Role{Name: domain.Admin}

    }

To be done

  • Assets versioning 
  • Member page is not informational to others than admin user, don't show it to others 
  • Verification of new user with email 
  • Forgot my password functionality 
  • Running with Raspberry pi would be nice, should be doable (http://www.mccarroll.net/blog/v8_pi2/index.html)


Related blogposts

Revisited: Isomorphic React.js with Go backend

sunnuntai 18. lokakuuta 2015

Using New Relic with a Go web application

I stumbled into a great middleware named gorelic.

It's too easy to start using it and you get useful information about your app.

package main

import (
    "github.com/labstack/echo"
    "github.com/syntaqx/echo-middleware/gorelic"
)

func main() {
    e := echo.New()

    // Attach middleware
    gorelic.InitNewRelicAgent("YOUR_LICENSE_KEY", "YOUR_APPLICATION_NAME", true)
    e.Use(gorelic.Handler())

    e.Run(":8080")
}

Memory usage

For example from the image below, I straight away knew that handling the right to view data for user with errors, seems not to be the right way. On such a page the response time rises immediately (green and purple lines).


Response time



perjantai 18. syyskuuta 2015

Revisited: Isomorphic React.js with Go backend

I found an interesting project at github named selfjs (https://github.com/nmerouze/selfjs). It's a small project, but it uses an other project named v8worker (github.com/ry/v8worker). The idea is great, to use chrome's very own v8 engine to interpret javascript. Selfjs even claims it's faster than with node which also is built on v8. This is of course server side rendering we're talking about.

Earlier I wrote about otto, and didn't think it was fast enough. But v8worker just might be. I made an almost equal application with selfjs and v8worker.

http://isomorphic.uutispuro.fi/ (using otto)
http://isomorphic2.uutispuro.fi/ (using v8worker)

The first page rendering takes about 0.27 seconds on my Mac. With a little caching that should be liveable.

source code
https://github.com/jelinden/go-isomorphic-react-v8

lauantai 21. helmikuuta 2015

Setting a cookie with Golang

Couldn't find a good example about how to set a cookie to response with Golang using Gin.

So, here is what I did

r.LoadHTMLTemplates("templates/*")
r.GET("/en", func(c *gin.Context)
 
{
    expire
 
:=
 time
.Now
().AddDate
(
1
,
 
0
,
 
1
)
    cookie
 :=
 http.Cookie
{"cookieName", "cookieValue",
        "/", ".domain.com", expire, expire.Format(time.UnixDate), 
        41472000, false, false, "cookieName=cookieValue", 
        []string{"cookieName=cookieValue"}}
     http.SetCookie(context.Writer, &cookie)
     c.HTML(200, "index.html", gin.H{"title""Main website"})
})


From documentation (http://golang.org/src/net/http/cookie.go)

type Cookie struct { 
    Name string 
    Value string 
    Path string 
    Domain string 
    Expires time.Time 
    RawExpires string 
    // MaxAge=0 means no 'Max-Age' attribute specified. 
    // MaxAge<0 means delete cookie now, equivalently 'Max-Age: 0' 
    // MaxAge>0 means Max-Age attribute present and given in seconds 
    MaxAge int 
    Secure bool 
    HttpOnly bool 
    Raw string 
    Unparsed []string // Raw text of unparsed attribute-value pairs 
}