• Everything We Wish We’d Known About Building Data Products – First Round Review
  Quote: "Where to Start Building: A lot of people choose to start building by modeling the product in question. Some start with feature discovery or feature engineering. Others start with building the infrastructure to serve results at scale. But for Belkin, there’s only one right answer and starting point for a data product: Understanding how will you evaluate performance and building evaluation tools. “Every single company I’ve worked at and talked to has the same problem without a single exception so far — poor data quality, especially tracking data,” he says.“Either there’s incomplete data, missing tracking data, duplicative tracking data.” To solve this problem, you must invest a ton of time and energy monitoring data quality. You need to monitor and alert as carefully as you monitor site SLAs. You need to treat data quality bugs as more than a first priority. Don’t be afraid to fail a deploy if you detect data quality issues."
  (categories: data quality testing bigdata linkedin twitter information )

• Dan Fredinburg | LinkedIn
  Died in the Nepal quake, quote from linkedin: "Additional Honors & Awards Boeing’s Air Force Systems Team of the Year Award 3rd Place – 2005 Took 1st Place in Google’s DWH/BI Lasertag competition – 2008 Won 1st Place in a Foosball tournament (teams) – 2009 Set Pop-a-shot Basketball record score of 157 – 2009"
  (categories: life love death values awesomeness kicking-ass )

• Dropwizard JDBI | Dropwizard
  Quote: "If you’re using JDBI’s SQL Objects API (and you should be), dropwizard-jdbi will automatically prepend the SQL object’s class and method name to the SQL query as an SQL comment…" OMG that’s really useful if you happen to be like me and have spent too much time in Postgres slow query logs.
  (categories: postgres database dropwizard java tips )

• Meet DJ Patil: Obama’s Big Data dude
  Quote: "In each new feature, Patil drove home the idea that the best sign of a good data product is no obvious evidence of the data itself. “The user doesn’t want to see raw data, they want the data in a usable form, and that usable form should help them do something more creative, be more efficient, give them superpowers,” he said. “Something you could never conceive of before.”" What a great story too.
  (categories: data visualization value problem-solving grit government )

• A Conversation with Werner Vogels – ACM Queue
  On services before microservices were a thing. Also note that this was written in 2006(!) and that Jim Gray (since lost at sea) did the interview: "The traditional model is that you take your software to the wall that separates development and operations, and throw it over and then forget about it. Not at Amazon. You build it, you run it. This brings developers into contact with the day-to-day operation of their software. It also brings them into day-to-day contact with the customer. This customer feedback loop is essential for improving the quality of the service."
  (categories: amazon soa microservices wernervogels )

• Don’t Take Micro-Services Off-Road | Dejan Glozic
  Quote: "If you carefully read ‘monolith to micro-services’ blog posts, you will notice that the end result is the same thing. Groupon team has not created a ‘catalog of social coupon services to be assembled into coupon applications’ – they rebuilt Groupon Web site. They broke the monolith into small pieces and rebuilt it again. As far as their end users are concerned, the monolith is still there – the site was rebuilt in mid-air."
  (categories: microservices architecture soa groupon paypal soundcloud netflix )

• EFF creates the "Mark Cuban Chair to Eliminate Stupid Patents" | Electronic Frontier Foundation
  Awesome.
  (categories: patents software law innovation )

• Prioritization and Value Maximization | Tyner Blain
  Math applied to the effort / value matrix.
  (categories: agile planning value strategy )

• 8 Questions You Need to Ask About Microservices, Containers & Docker in 2015
  Great list of issues that have to be addressed if / when you use microservices.
  (categories: microservices docker )

• CTO Chris Satchell on Nike’s Continuous Delivery Strategy — Heavybit Blog
  A couple of good nuggets here as we move down this path: a) Quality is not a gate, it’s an ongoing operation b) Make Release Management about process verification: verify the process for release, not the release itself.
  (categories: continuous-dev nike devops microservices )

• District Data Labs – Modern Methods for Sentiment Analysis
  Seems like the clustering of similar words could be used for some types of spam analysis against community software assuming you had a corpus / dictionary of words commonly used in the community to check against.
  (categories: spam clustering ngram algorithms nlp classification )

• Dev and Ops Collaboration and Awareness at Etsy and Flickr
  old, but yet another OODA reference.
  (categories: devops flickr etsy monitoring continuous-dev )

• Apocalyptic Schadenfreude — Matter — Medium
  Quote: "…the vast majority of people benefiting from that water don’t live in California at all. The Central Valley takes up only 1% of the landmass of the United States, but it produces 25% of the food we eat, and almost half of the fruits or nuts we consume. California is running through its water supply because, for complicated historical and climatological reasons, it has taken on the burden of feeding the rest of the country."
  (categories: environment water california food )

• How LinkedIn Execs Run Meetings | Brian Rumao | LinkedIn
  who, what, why applied to meetings.
  (categories: meetings productivity prio )

• Post-Mortems at HubSpot: What I Learned From 250 Whys
  Some great points here: – Let’s Plan for a Future Where We’re All As Stupid as We Are Today – Less Root Cause, More Broadest Fix – Err on the Side of MTTR, Not MTBF – Never Let "Slow Down" Be The Answer
  (categories: failure management software root-cause-assessment rca problem-solving )

• 4 reasons why microservices resonate – O’Reilly Radar
  Quote: "One of the reasons we talk about “microservices” instead of just “services” is the unit of change. Traditional service-oriented architectures are typically designed to maximally leverage shared resources. The side effect of a shared resources design is a lot of physical coupling, where each service shares a monolithic database, an object-relational mapping layer, and lots of shared implementation elements. Architects are also drawn to single sources of truth to eliminate duplication, which encourages “smart” architectural elements like Enterprise Service Buses to handle chores like transformation and orchestration. If you look at SOA architectural goals, the desire is to encapsulate behavior and secondarily share resources. However, traditional SOA architectures make isolated change hard: when everything is highly coupled at the implementation level to share resources effectively, that coupling hinders isolated change. Microservices strive for integration coupling but no physical coupling. The goal is both encapsulated behavior and ease of operational change."
  (categories: microservices soa architecture devops )

• Meat Production Wastes Natural Resources | Animals Used for Food | The Issues | PETA
  It takes more than 2,400 gallons of water to produce 1 pound of meat, while growing 1 pound of wheat only requires 25 gallons. You save more water by not eating a pound of meat than you do by not showering for six months!
  (categories: energy water food )

Last Friday was a “Spring Holiday” at Jive but all my little dudes were in school so I figured it’d be a great day to go into work and actually try to do something technical for a change of pace. My oldest has become completely absorbed in Minecraft and offered up that he’d really like to learn how to program mods a couple weeks ago so we bought the Kindle version of this book, which suggested the use of Bukkit. Simultaneously, I really wanted to get my hands a teensy bit dirty with Docker so I spent the time on Friday trying to get the two to place nicely together so that I could deploy a Docker image of Bukkit to my hosted server. Sidenote: I’m well aware of the fact that there are tons of Docker images already built for Minecraft, I personally learn faster and better by doing rather than reading.

My personal laptop is running OSX so the first order of business was getting Docker installed there. Docker doesn’t actually natively run on Macs so I installed boot2docker, which I think is a little bit Inception-ish (VM inside of a VM?) but seems to work just fine. After that I ran through this tutorial:

https://www.docker.com/tryit/

which was / is really well done. Not sure if there’s a tool that they used to build the interactive console but either way, great work on their part. I got my Docker.io account setup (https://hub.docker.com/u/ajohnson1200/) and then started down the path of trying to create a Docker image for Bukkit.

As I was investigating Bukkit as a development server I came across this presentation:

which forced me to call an audible at the line, didn’t look like Bukkit was going to be a valid platform to develop on. Turns out that the book I mentioned above has a second version, which instead of using Bukkit, uses CanaryMod, which does allow for custom Java plugins and doesn’t have licensing problems like Bukkit.

After going through the tutorial (which was really basic), my first thought / approach was to spin up a simple image so I started using this one:

https://registry.hub.docker.com/u/tianon/true/

and then using the console to install packages into that and to then save and publish it. But that was a rookie move, Docker has a much better way of creating images via Dockerfile, where you write a recipe for the image that you want to create and then let Docker build that from the recipe. So then I spent a bunch of time looking at some of the existing Minecraft (Bukkit or CanaryMod) Docker images that exist in the wild, examples:

https://github.com/mbrevoort/docker-minecraft-bukkit
https://github.com/tclavier/docker-scriptcraft
https://github.com/chrisabrams/docker-minecraft-with-bukkit

all of which were helpful. Became obvious that I’d use a Dockerfile and some kind of startup script and so after thinking about everything for a bit I ended up with a Dockerfile that looks like this:

from dockerfile/java:oracle-java8
env DEBIAN_FRONTEND noninteractive

volume ["/minecraft/"]
cmd /start
First line indicates that I want to base my image on the Java8 image (right?), second line puts the OS into noninteractive mode as documented here. Next I take the jar file that I downloaded from the CanaryMod site and add it to /opt/minecraft and alias my startup script to /start and give it the proper permissions. I create a mount point using the volume instruction which I’ll use later as the place that CanaryMod will start it’s data and then finally a cmd instruction to initialize the container, which leads to… a startup script that looks like this:
#!/bin/bash
set -x
set -e
cp /opt/minecraft/canarymod.jar /minecraft
cd /minecraft

/usr/bin/java -Xmx4096M -jar canarymod.jar
First two lines I copy / pasted from another startup script without knowing what they did but just read up on them via this nice blog post so basically we have some nice defaults for the startup script. Next, I copy the jar from the location that it exists in the image into the volume defined in the Dockerfile and then startup the app on the last line. When CanaryMod starts up, it’ll automatically create a bunch of directories and a couple of files:

canarymod.jar config db dbadapters eula.txt lang logs pluginlangs plugins usercache.json worlds

one of which you’ll need to change (eula.txt) before it’ll complete the startup process. I left that as a manual process that you’ll need to do if you use my Docker image (which you shouldn’t, mine is very basic). Once I had those written I built the image:
docker build -t mycanaryapp .
and then I decided that I didn’t like the name so I changed the name:
docker tag mycanaryapp:latest ajohnson1200/canarymod:latest
and then I pushed it to the Docker repo:
docker push ajohnson1200/canarymod
After that, I logged into my hosted server (which has been running for years on Rimuhosting) and attempted to install Docker. Docker wouldn’t work with the kernel I had installed so I had to update that and then got Docker installed in a couple minutes, pulled my image:
docker pull ajohnson1200/canarymod
and then started up the container:
docker run -v /usr/hosts/canarymod:/minecraft -p 25566:25566 --rm ajohnson1200/canarymod
binding the external port 25566 to the same internal port since I another Minecraft server already running on 25565 although now that I think about it, I think I could run CanaryMod inside the container on 25565 and just bind the external port to 25566 and not have to change the config that lives inside /minecraft. I’ll have to check that out.

Not sure if I took the right approach or not but the big thing that I didn’t spend time thinking about was storing data inside of the container instead of persisting the data on the volume outside of the container. I *think* that they way I’ve done it actually makes it possible to continually update the CanaryMod distribution while not touching the data that it stores, which is nice.

All in all, a fun exercise and I learned a bunch. If you’re looking for a CanaryMod image, don’t use mine. 🙂

Other links:

• http://poor-impulse-control.herokuapp.com/?page_id=6221
• https://registry.hub.docker.com/_/java/
• http://zeroturnaround.com/rebellabs/docker-for-java-developers-how-to-sandbox-your-app-in-a-clean-environment/
• http://blogs.atlassian.com/2013/06/deploy-java-apps-with-docker-awesome/