Friday, January 03, 2014

Few handy things to know in Go Library (Command line arguments, Reflection, Setting Bits, Hash functions)

Command line arguments in Go

In order to access the command line arguments passed to a Go executable, we work with the "flag" package from the standard Go library. Shown below is an example of passing command line arguments.
go run cla.go Krishna
Or if you have an executable obtained from go build, you can run it as
./cla Krishna 
Shown below is an example go snippet that accesses the first argument passed & prints it.
package main

import (
    "fmt"
    "flag"
)

func main() {
 //Without the call to Parse(), arguments may not be accessed
 flag.Parse()
 fmt.Printf("%s\n",flag.Arg(0))
}
Notice that you will have to invoke the call to flag.Parse() without which you will not be able to access the arguments.

Reflection support in Go

We take the previous example, add some code for reflection which will help us understand how reflection library may be used. Just some simple example without having to go deep. For that, read this post from the official Go blog.

So what is the type returned by flag.Arg(0) ? We will start from there and go use some functions.
package main

import (
 "fmt" //regular format package
 "flag" //package from command line arguments
 "reflect" //package for reflection
)

func main() {
 //Without the call to Parse(), arguments may not be accessed
 flag.Parse()
 fmt.Printf("%s\n",flag.Arg(0))

 passed := flag.Arg(0)
 //TypeOf actually returns a Type type
 fmt.Println("typeof : ",reflect.TypeOf(passed))

 //ValueOf actually returns a Value
 fmt.Println("valueof : ",reflect.ValueOf(passed))

 //Example functions from the Value type
 fmt.Println("type: ",reflect.ValueOf(passed).Type())
 fmt.Println("kind: ", reflect.ValueOf(passed).Kind())
 fmt.Println("interface:", reflect.ValueOf(passed).Interface())
}

Big Integer & Set Bits

In the future post, I would like to implement a Bloom filter in Go but for that I will need to know how to do the following:
- Create a big number and set some bit "x" to 1 or 0.
- Accept a string, convert it to byte array, then hash the byte array.

To support the above, I wanted to see how simple big numbers would work. I am only looking enough to support my purpose. 

Big Numbers (multi-precision numbers) are supported using "math/big" package in Go library. Shown below is an example code that does that.
package main

import ( 
   "fmt"
   "math/big"
   "unsafe"
)

func main() {
 
 //create a new big integer
 filter := big.NewInt(0)

 fmt.Println("filter value: ",filter)
 
 //accessing the sizeof from unsafe package
 fmt.Println("size of : ",unsafe.Sizeof(filter))
 
 //example of how to cast uintptr to int
 filter.SetBit(filter,int(unsafe.Sizeof(filter)*8)-1,1)
 
 fmt.Println("filter after setbit last to 1 : ",filter)
}

The above example shows a few things:
- Creating an instance of big.Int using the NewInt() call.
- Accessing the size of the big.Int using unsafe package exported SizeOf() function
- SizeOf() returns uintptr and we convert that to integer - that is a good example of casting
- How to set bit on the big.Int using SetBit

Working with Hash functions

Example below demonstrates the following
  • how to convert a string to a byte array
  • how to use FNV Hash from the standard Go library
package main

import (
 "fmt"
 "hash/fnv"
)

func main() {
        //Create a hash fnv function
 hash := fnv.New32()
 
 name := "Krishna"

 //converting string to byte array
 namebytes := []byte(name)

 fmt.Println("hash reset : ",hash.Sum32())
 
 hash.Write(namebytes)

 fmt.Println("hash of Krishna : ",hash.Sum32())

 hash.Reset() //resets the hash

 fmt.Println("hash reset : ",hash.Sum32())

 //write again
 hash.Write(namebytes)
 
 fmt.Println("hash of Krishna : ", hash.Sum32())
}
Hash libraries in the Go "hash" package implements the Hash interface. The implementations in the standard Go library as of this time are adler32, crc32, crc64, fnv-1/fnv-1a. The above example creates an instance of FNV hash, writes a byte array to it and then gets the value through Sum32() call. You can "forget" what is written to it by Reset() function which will allow you to reuse the hash instance.

I am newly learning Go and along the way sharing some things that I learnt. If you know of a better way to accomplish what I have shared here, please feel free to comment.

Wednesday, January 01, 2014

Working with "Packages" in Go language

Packages in Golang

The specification states that all source files for a package be stored in the same directory.
You can import a package as shown below.

import "fmt"

In this case, the Printf method may be accessed as fmt.Printf(). All other identifiers exported by the package is also accessible as fmt.XXXX().

You can also do the following.

import f "fmt"
In this case, the qualifier is "f" instead of the default "fmt". So you access as "f.Printf()".

One other way is the following.
 import . "fmt" 
In this case, you dont need a qualifier for the exported identifiers from the fmt package. You may also use the blank identifier (underscore : _) as the qualifier name for cases where you wish the initialize the package but not necessarily import its exported identifiers.
Source files of a package have an init() function. See below for more details.
The previous example of a simple Web Server is now split into two files
//GoWebServer.go
package main

import f "fmt"

//init() is a special function that gets initialized.
func init(){
 f.Printf ("Package Initialized\n")
}

func main() {
 f.Printf("Web Server will be started at port 8080\n")
 runWebServer()
}
//WebServerHelper.go

package main

import (
 . "fmt"
 "io"
 web "net/http"
)

func init(){
 Printf("Initializing WebServerHelper.go \n")
}

func runWebServer(){
 rootHandler := web.HandlerFunc(func(w web.ResponseWriter, r *web.Request) {
  io.WriteString(w, "Hi there, I love "+r.URL.Path)
 })

 srv := web.Server{
  Handler: rootHandler,
  Addr:    ":8080",
 }
 srv.ListenAndServe()
}
Few things that I learnt when trying to make this work.
- You need to specify all the files required for your main program to build properly. So you may
go build
Or you may run
go build *.go
- You can also specify output to be generated as
go build -o output/gowebserver
Or be more explicit
go build -o output/gowebserver *.go
- Go complier does not like if a package is imported but not used. This is good. I cannot tell how annoying i find C# using statements that are just there but no one really uses them.

Notice that both the files have an init() function defined and when I build everything and run the program, the output is as follows:
Package Initialized
Initializing WebServerHelper.go 
Web Server will be started at port 8080

Such init() functions may be defined multiple times even in the same source file & the execution order is unspecified/undefined. So we should not write our initialization logic based or the execution order.
And init() cannot be called by anyone else. Packages are initialized only once. So if you import packages P & Q but Q also imports P, then P is initialized only once.

Package initialization is comprised of variable initialization (package level variables are all assigned initial values) & invocation of init() functions - performed one package at a time in a single go routine.

See below for a basic program that indicates the above said sequence - variable initialization followed by init() method.
package main

import "fmt"

//x := 2000
var x int = 2000

func init(){
 var y int = x
 fmt.Printf("X value is %d\n",y)
}

func main(){ }

x is a global variable (which by the way cannot use the shortcut notation that is commented) that is initialized to 2000 and that is what gets printed when the program is executed.

Local Packages in Go

When you say
import "fmt" 
the package "fmt" is looked for in the standard Go tree. Then it is looked for in "$GOPATH/pkg" directory where $GOPATH is the environment variable set to some directory on your machine. Commands such as "go get" also looks for this directory. You can read more about it here (look for Go path). So what about local packages - the one you do not want to put in GOPATH but leave it in the current directory for your application? To make this work, I renamed the package declaration (which originally was main) for the WebServerHelper.go as
package ws
Then I created a folder with the same name as the package & placed the .go file in the folder. Then my import statements on the GoWebServer.go (the main file) has changed to
import (
 f "fmt"
 ws "./ws"
)
Notice the "./ws" declaration. So the import "XXX" statement says look for "XXX" file path. So without "./" it was looking in standard GO tree, then it looks at GOPATH. If there was a "./", it was just looking at the local directory. You may also give absolute paths. Also you don't need to build this directory specially. Just have your main file compile and you are good.

One other things that I had to change in the code to make the application work as before.
- In packages, if your function is to be exported, then it should start with upper case. In general, a field or a function or other identifiers within a package that starts with upper case letter are exported.

So if you are trying to make the previous code run (after the directory changes as described above), you will get the following

./GoWebServer.go:15: cannot refer to unexported name ws.runWebServer
./GoWebServer.go:15: undefined: ws.runWebServer

So just change the name of the function to "RunWebServer" & it becomes exported and ready to use.

That's it for now!