Go Modules and How to Migrate

I have just ported one of my internal modules to use Go Modules instead of dep and I ran into a interesting problem and some pleasant surprises I will like to share with you!

If you have a project running another dependency manager e.g. dep it is possible to make an automatic migration to go modules by doing the following:

$ cd $GOPATH/vcs/organization/project
$ export GO111MODULES=on    # To use modules on projects within your $GOPATH
$ go mod init

After this it is possible to use go module commands in your coding workflow such as go mod tidy which will tidy-up in your dependencies based on your code.

Gotcha with Module Name and Internal Packages

When doing this automatic migration $ go mod init will call your module the path from $GOPATH e.g. bitbucket.org/organization/project.
However, when I did this automatic migration I did it on a clean clone of the project.
But I did not place the project within the correct path and my module name was set to module project.
When doing go build it will invoke go get  to get the internal dependencies placed within internal/ which fails with the following errors:

go: missing Mercurial command. See https://golang.org/s/gogetcmd
go: missing Mercurial command. See https://golang.org/s/gogetcmd
go: missing Mercurial command. See https://golang.org/s/gogetcmd
go: missing Mercurial command. See https://golang.org/s/gogetcmd
go: missing Mercurial command. See https://golang.org/s/gogetcmd

After a good amount of googling on the issue I found this: https://github.com/golang/go/issues/27224#issuecomment-416000753
So the fix is to manipulate go.mod from module project to module bitbucket.org/organization/project and everything started working correctly.

No more $GOPATH!

Now you can move your project to where ever you want on your machine and still go build, test, and run as you normally do!
E.g. try and move your project to /dev/project and execute a go build
This is such a nice change I cant even find the words to describe it!


It was purposed to totally remove vendoring (you may know it as the vendor directory) in go modules and I actaully thought it would get removed. But the community had good reasons and it was decided to keep it!
You can read more here: https://github.com/golang/go/wiki/Modules#how-do-i-use-vendoring-with-modules-is-vendoring-going-away

If you like vendoring (and not just ignore it in your .gitignore) you can keep using it by doing:

go mod vendor

This will create a vendor directory including all your dependencies.


In conclusion I ran into a gotcha, because of a human mistake, but overall I really enjoy modules and I think the go team have done a fantastic job especially with the migration from prior dependency managers such as dep:  https://github.com/golang/go/wiki/Modules#automatic-migration-from-prior-dependency-managers
I will definitely port more projects to go modules in the future and most importantly no more $GOPATH!

Prometheus 101 – How to get started

I recently gave a talk about Prometheus and how to get started at Cloud Native Aarhus Meetup #6.
Cloud Native Aarhus is one of the most fantastic communities I have been a part of!
The shared vision and love for cloud native by the people is just staggering.
If you wanna join us and talk about everything Cloud Native you can sign up at our Meetup page or join our slack and come chat with us!


The talk was named “Prometheus 101 – How to get started” and targets how to get going with the most awesome monitoring solution out there.
It is important to understand the components within Prometheus and the architecture of the complete system which can be found here.

The demo repo is open source and available here: https://github.com/steiniche/prometheus.
Thanks to vegasbrianc for the original repo and idea.
One of the cool things about the demo repo is that it only requires docker and docker-compose to try it out!
The day after the meetup Grafana 5.0 was released which the repo now uses and it works without any hassel on your part.
I received so much positive feedback about the presentation, so if you were at the meetup and you are reading this I want to say thank you very much, I really appreciate it.

Maybe there will be a Prometheus 201 talk some day where we deep dive into the awesome details about this fantastic monitoring system.
One of the things I think people should dive deep with is PromQL which is the query language used by prometheus.
Another thing is the service discovery capability which works out of the box with container clustering management systems such as Kubernetes.
Until then I hope to see you out there at the meetups here in Denmark and maybe hear your awesome talk!

Ansible vs Docker

Ansible vs Docker in the delivery pipeline.
I believe there is place for both Ansible and Docker and we have something to show for it! In my current team one of our goals are: all our infrastructure as code in git. There is many ways of doing this but we are currently using Ansible And Docker. There is not many resources on google telling you how Ansible and Docker can benefit each other. But Ansible’s own site actually have three key points about this: https://www.ansible.com/docker. These key points are: Flexibility, audibility, and ubiquity.


Ubiquity is what makes a combination of Ansible and Docker viable for us. Even though containers are awesome they cannot be run in total isolation. What I mean by this is that before we can run docker containers on our machine we need to install docker itself. There can be requirements about how the network of the machine should be setup. Sometimes we need to configure security aspects on the machine e.g. firewalls.
We are running a setup without a docker hub instead containers are distributed to the machine from a central tool: Ansible.
Ansible vs Docker
An easy analogy is the one the image is showing: Ansible prepares and moves containers, and the containers contain the application.

Example workflow

Our environment is a mix of windows and CENTOS boxes. We are currently not running docker on the windows boxes.
But by having Ansible we can actually configure both windows and linux system from one central place.
An example workflow from our setup:
Linux: Provision machine for docker -> move docker container containing web application to the machine -> run container.
Windows: Provision machine -> configure it for a web application -> deploy web application.

By using Ansible we can have a central place for both provisioning and configuration all of our machines. We are currently using powershell for windows and python for linux, however Ansible is equipped with a fairly large set of core modules which can fulfill most requirements. We have found Ansible easy to implement into our tool chain and all in all easy to work with. You can really feel that this is a tool by developers for developers doing operations.
There is a point to be made about kubernetes, and the like, where you have a platform for running your containers on a set of kubernetes hosts. But you would still need to configure kubernetes and what better tool to use than Ansible!
To summarize if you have a goal like ours, all infrastructure as code, Ansible and Docker is a great tool combination for making it a reality.

Moving a folder from one git repository to another

This will be one of the more technical posts and I really think this could help someone else.
I think it is a very normal situtation to be in: you have grown out of your one repository to rule them all.
And you know that you are not google.
This post will explain moving a folder from one git repository to another and the amazing tool: git filter-branch.
I had a pretty interesting task the other day, namely moving a folder from one git repository to another.
The folder was ready to live by it self as it had grown its own eco system with a complete test suite and did not have any ties to the main repository.
There were just one condition: We did not want to lose the history regarding all the files in this folder.
I got the task of figuring out how we could move the content and history and luckily for me I am not the first one try to tackle this problem.
I pieced the gist below together from various blogs and stackoverflow posts but this stackoverflow post helped me the most. 
Enough rambling, lets look at some git commands.

To the git commands

Here be dragons!
I think git is an amazing tool and you should be able to do this kinda crazy thing as well, but please be careful!

The interesting part in the gist above is: git filter-branch --subdirectory-filter node/ -- --all
Which can actually do the one thing we normally do not like in Configuration Management: rewrite history.
However, in our case it is actually what we want, rewriting the git history that is, because it will trim the repository to only contain the content and history of our node/ folder.
However, please be careful when using git filter-branch as it will infact rewrite history and if you push it to your master branch on the wrong repository your gonna have to revert.
Hope this helped you out in your git endeavors.

Startup ideas

This post is about startup ideas and why sharing yours is a good idea!

I am active in the young startup community here in Denmark and one of the things I have been seeing lately is people searching for brilliant tech people, but they will not tell anything about their idea for their new startup.
I think this is downright wrong and here is why.
I am not suggesting that you share your entire business model, but share enough to get your crowd listening.

Your Startup – Your team
When you run a startup you want the best people possible to join your team, because this will give you the biggest chance of success.
However, the best people don’t come easy, they can be hard to find, and they need to be conceived that your startup will succeed or at least have a very high chance of success and reward.
No one wants to join business which is dead from the get go.
Your idea is what started it all, and what made you believe in a startup, so tell it to the crowd, and tell it loud.
Don’t just tell it to a few people at a time, tell it in the forums where the brilliant tech engineers are.
Most people have a sence that they have lost their idea and someone will steal it when they tell it.
However, as I see it, they just got a ton of feedback saving them valuable time to market.
Furthermore, it can help you hook partners and investors because you make them believe in your idea as well.
When working in a startup you have scares resources and little time, don’t waste it telling your idea to one person at a time.

Execution and timing is everything

If we listen to Bill Gross execution and timing are the two most deciding factors when it comes to success for a startup.
The idea only gets an 3th place.

And when you are the founder of a startup your most definitely want your startup to have the best probability for success.

Furthermore, we are in a time where the product starts to beat marketing.
It is more important to create a product which your customers want than telling them that it is what they want.
In these times it is fairly easy switching companies for the best product in the market.

Your followers
Lastly you will have the early access effect.
This is something very popular in the game scene right now where you give out early access to a few people such that they will tell all their friends that your game is awesome!
You have the possibility of getting followers which will tell their friends that your product is awesome and they should join as well.
Having such followers or fans is crucial for making it in today’s markets from my perspective.

The conclusion is spread your idea, be ready for it to morph into something even better, be challenged by people and make your product better!
Share your idea with the world and share it loud!

Devops – what is it and why should you care?

I must admite that I do not admire the term devops but it is a term that seems to stick these days.
The term describes how developers and other IT professionals, specificly IT operations, should communicate and collaborate more effectively with each other.
However the term is now used to describe a special role in software development, namely the developer which also knows about operations.


Devops consists of three main areas: Development, Operations and QA

I am currently in such a job where I am doing a great amount of devops work to keep my team even sharper and faster at developing great software which brings value to the customer.
From my perspective devops is about minimizing the ops and maximizing the dev.
Ops or operations is often tedious tasks, requiring manual labor, which needs to be performed with precision.
This is a lot like testing which we as software professionals automate as much as reasonably possible because it repay us with fast feedback loops and the certainty of quality with out all the tedious work!
As a side benefit we can avoid many of the human mistakes, which we all make from time to time.
Devops is about automating the operation aspect because of the same reasons, eliminating human mistakes, be certain about quality, delivering business value to the customer fast, and saving resources.
It is actually that simple!

However I currently see some of the players in the market being afraid of investing in devops related improvements and I believe its because they are yet to see that it will bring value to them and their customers businesses.
The story about how Knight’s Electronic Trading Group (ETG) lost nearly $400 million in about 45 minutes, where the fault was placed with their deployment procedure, gives a great perspective into why devops and creating delivery pipelines is so important.
If you get it right you will get all the benefits above, if you don’t you will have to rely on your employees not making mistakes, which per definition they will, because it is human to make mistakes!

Visions and software innovation

In Essence, an innovative methodology, the metaphor used to describe an idea is called a Vision.
Visions are used to keep an eye on the overall goal and are a way for the team to describe and communicating about an idea.
They can be used to discover and communicate about limitations and challenges of an idea or possible solution.
A representation of a vision can take different forms, which can be seen seen as two axes, concrete versus abstract, and simple versus complex.
This means that it is possible to have four distinctive representations:

Essence vision types and representations

Essence vision types and representations

  • Concrete means that the vision is material.
  • Abstract means that the vision is immaterial.
  • Simple means that the vision is uncomplicated in form.
  • Complex means that the vision consists of interrelated parts or aspects.

In Essence each of these representations have a name:

  • Icons (concrete simple)
  • Prototype (concrete complex)
  • Metaphors (abstract simple)
  • Proposition (abstract complex)

The software industry uses these representations extensively today.
E.g. prototypes are produced each day and they are used to communication about properties of a product.
Icons are used for communicating about one specific property e.g. it needs to be more simple in use, like an apple remote.
As software professionals we often use metaphors for communicating about code, and likewise can they be used to communicate about a vision for the product being produced.
Propositions is a more complex “metaphor” often being a hole sentence describing the solution to a problem.
An elevator pitch can be seen as a proposition.

I have used them all with success both to investigate but also to communicate about an idea with stakeholders and other team members.
Prototypes is the representation I use the most.
Prototypes works well for testing practical aspects, however they are very concrete and can be complex to comprehend. The problem with them being concrete is that they can limit creativity and users can come to think that this is the final product.
This is not always ideal as the customer participation is needed in discussing the product on a level where everything is possible and where the team is looking for opportunities.
However, in the Pay-E-Safe’s Zebra project we used prototypes for presenting our vision and using this presentation it was possible for us to investigate the use context for the solution.
The main customer based was located in Uganda, Africa where there were a need for a payment solution as Zebra.
By choosing a representation which could be interacted with we learned a lot about how the customers in the domain would use the solution and what was important to them.
For instance we initially figured that a pin code would be to tiresome and slow the purchasing process.
However, they thought that pin codes were important because they felt that this protected them against theft.
This goes to show that representations of visions, and the choosing of which to use, is important for the investigations, the innovation of the solution, and for bringing value to the customer.
Put in other words visions can be used to understand the domain, but also for keeping a long term strategy for the product.

Understanding the domain

In software engineering understanding the domain is key to providing business value.

“The design of good houses requires an understanding of both the construction materials & the behavior of real humans” — Peter Morville

To develop good, comprehensive software we, as software developers, need not only to understand the technology and architecture behind a given solution.
But we need to understand the business domain including the customers needs.
Many great projects have been discarded because of poor domain knowledge even though the solution is a shining master piece with all the bells and whistles.
A correction; the solution is not a master piece until it can support the customers business.

The business domain is of highest importance when developing a software solution.
Customers will often happily purchase minimum viable products, if it support their business needs.
I.e. if it do not it is of no value to them, even with all the bells and whistles.
Providing business value to the customer is a tough nut to crack and it can never be considered trivial.

“Customer collaboration over contract negotiation” – Agile Manifesto 2001 Kent Beck et. al.
The manifesto describes how it is important to understand the customer which we are creating software for, and that is not done by writing comprehensive contracts.
Software is one of the business areas where everybody on the team needs an understanding the domain, it is hard to define everything beforehand and therefore, we use techniques which let us do a little at a time to comprehend what we are building.
It requires understanding of the materials used and behavior of real humans!

Does the customer know it all?
Certainly not, when having a successful product the strategy changes and it can be good not to listening to customers and instead focus on disruptive innovation to stay competitive in the market.
However, this will be left for another blog post.

Everybody on the team need to care!

This post springs from something i see very often in projects I am involved with, namely that if everybody cares the team will reach a higher state and produce better products.

Caring – a long and painful road
This may sound simple and easily overcome but the reality is quite different from my observation.

If everybody on a team cares about whats being produced I promise you the projects goals will be fulfilled, and provide greater business value.

However getting there is a very long, and sometimes painful, road.
If team members do not care and let something slip it risks the hole project – and risk is something to be mitigated not created.
The team members not caring about the product is essentially dead weight.
Weight which the other members of the team needs to carry to reach goal – namely provide an amazing product!

Caring about something is a very powerful emotion, it makes people persuade others about politic views, religions, or apple products!
Image a team that cares so much about their product that they just want to tell everybody about how amazing its gonna be when it hits the streets.
If the product hits a road bump everybody will care so much about fixing the problem that they will use all of their knowledge, time, and resources in collaboration to get the product in that amazing state once again.

This team will also be the most beneficial seen from a business stand point.
The management behind the team can be 100% certain that the product will be produced with all the love and nurturing a software project needs!
And god help the individual which hurts the project, because the team will fight with tooth and claw to protect and ensure the product well-being.

If everyone in the team cares about every important aspect of it has a much higher probability of become a success and hereby becoming a profitable business.

Creating the caring team
First of all you need to ensure that you have team members able of caring.
If the team can work together and have the needed knowledge, skills, and resource coupled with caring they are likely to going the extra mile for the project.
For this to happen the team needs to have a clear view of the goals of the project – namely a vision.

Furthermore the team needs transparency, everyone in the team needs to know that each team member is caring there own weight and not just wasting their time waiting for the clock to reach 5pm.
If this transparency is not ensured it can be hard for team members to feel that everybody is doing their absolutely best possible to make this product amazing.

To create this transparency it is important to match expectations within the team.
This ensures the team have the same goals for the product and what the product needs to stand for.
This also highlights if some team members thinks it is okay to go for mediocre and not awesome.
Matching of expectations should be one of the first actions done within a team and should be a reoccurring event.

Caring dangerously
Caring, if not managed the right way, can lead to gold plating, analysis paralysis, and over designing.
However this can be minimized by ensuring transparency of the business aspects, because the product is not going to be amazing, from the customers view point, if it is not on time and within budget.

Everybody on the team needs to care about the product under construction.
A caring team, with the right knowledge, skill, and resources, will produce amazing products.
Amazing products which provides business value for the customer will inevitably generate profit.

So can you afford not to have a caring team?


Metaphors are a part of our everyday lives as software professionals. We use them to describe concepts, features and even complete systems to each other on a daily basis, but what are they exactly?
Kim Halskov Madsen wrote an article in 1994 called “A Guide to Metaphorical Design” and I think this is worth a read for everyone interested in improving their ability to describe concepts that may be hard to grasp e.g features to customers, a complete system to your boss, or just a newly build class to a fellow colleague.
One of my personal favorite metaphors is The Chicken and the Pig and how these two animals can describe commitment in a project.

How to use metaphors

The article explains how metaphors should be generated, evaluated, and developed.
A metaphor should, if possible, build on already existing metaphors e.g. something already existing in the problem domain.
If a predecessor artifact is already described this can be used as a metaphor e.g. files in file cabinets can be used to describe file systems on computers.
Rich structures are important for metaphors because general or abstract metaphors can become a problem e.g. a link can mean a lot of things, a path from somewhere to another, something describing a relationship between a source and a destination etc.
One should always evaluate the applicability of the structure chosen for a metaphor: does it really cover all the important aspects? Or can it be seen as misleading?
This also relates to the fact that one should use a metaphor that is suitable to the audience. I reckon that this is the reason why business and software people often talk past each other. They do not communicate in a language which they both understand.
E.g. a pointer metaphor is fully understood amongst software developers, but business personnel can have a hard time figuring out what a pointer is and what it represents.
Metaphors should be chosen with well-understood literal meanings, as the key aspect of metaphors is explaining something in terms of something else.
Metaphors are used for seeing something in a different light; from another perspective. Therefore, metaphors should be chosen with a conceptual distance between the source and the metaphorical meaning. E.g. a credit card can be metaphorically expressed as an electronic wallet or pocket.
It is important to have one or more bridging concepts contained inside the metaphor e.g. how the pig and the chicken commit to making brunch! In this example project, commitment and cooking form the bridge.
Elaborate assumptions – it is important to explicitly express what the metaphor hides and what it highlights.
Metaphors often use triggering concepts, e.g. “to meet the customer in a new way”, and these are necessary to elaborate upon.
Identifying the unused parts of a metaphor, e.g. aspects, features, and properties, in the source domain, can be used for gaining a more profound understand of the target domain.
Talking about the target domain as if it was the source domain, can be used for understanding how the target and source domain is intertwined.
Metaphors are helped by psychical structures in everyday experiences and can be used for describing abstract concepts in terms of concrete and familiar objects.

Metaphors can even be used for providing detailed, specific design options, and as a basis for justifying design decisions.
They can help people see something as something else and describe a model of the system which they can familiarize with.
Metaphors can shift the focus of attention and create a problem setting – they can provide a novel view of reality.

I hope this can help you understand how to use metaphors as the great tools they are – both for communicating with others, but perhaps even more with understanding problems at hand in a new light.