Docker registry auth issues

Docker registry auth issues

I ran into an issue today when scripting a setup for a private docker registry and I ended up wasting a few hours on it. I’m hoping this post can save a few people the same headaches.

After setting up a docker registry with authentication following most of what was in I was confronted with a rather annoying message on my console:

Error response from daemon: no successful auth challenge for https://registry:5000/v2/ - errors: [basic auth attempt to https://registry:5000/v2/ realm "Registry Realm" failed with status: 401 Unauthorized]

Unauthorized seemed pretty clear, so I checked the user’s htpasswd again, and recreated the htpasswd file:

htpasswd -bc /opt/docker-registry/auth/htpasswd username password

That didn’t work. I knew that there’s no way I messed up the username/password again, so I took a look at the output from the docker container hosting the registry:

time="2016-01-18T00:04:59Z" level=warning msg="error authorizing context: basic authentication challenge for realm \"Registry Realm\": authentication failured" go.version=go1.5.2 http.request.remoteaddr="" http.request.uri="/v2/" http.request.useragent="docker/1.9.1 go/go1.4.3 git-commit/a34a1d5 kernel/4.1.13-boot2docker os/linux arch/amd64" version=v2.2.1

The “authentication failured” type was actually extremely useful, it lead me right to the go source code that produced the error:


So, it looked like this was definitely a run of the mill auth error. After confirming the process could indeed read my htpasswd file, I took a closer look at the source code..


“Only bcrypt hash entries are supported”

…. CRAP! I forgot to use -B on my command line options for htpasswd..

htpasswd -cbB /opt/docker-registry/auth/htpasswd username password

Fixed my issue, and the world was lovely again!

The further consequences of free and cheap

The further consequences of free and cheap

I touched a little bit on this subject on the InWorldz blog in a post titled The hard thing about hosting things, but lately this has been coming up more and more, and I’m starting to get annoyed.

There are some very talented people in the OpenSim space. People with a wide range of skills from C++ development, C#, web development in PHP, node.js and other languages. The thing that all of them seem to have in common is that they have a passion for working on 3d software and simulations, but unfortunately we’ve found more and more that this passion and drive is being taken advantage of, and just like hosting prices in OpenSim, a lowest bidder mentality has emerged.

As a business owner and employee of small businesses, I learned a long time ago that when people don’t have to worry about paying their bills, they are more likely to be creative and solve problems in effective ways. They are loyal to your vision, and want to help see it through. They don’t have to seek out work elsewhere to make ends meet and they are appreciative of the sacrifices that you make for them.

Unfortunately the stories I’m hearing from people getting paid for OpenSim based work and from OpenSim grids paints a really disturbing picture of the way people are being used and led astray with promises.

The most recent I’ve heard is from an extremely talented software developer that is also serving as a systems administrator, a devops person, and someone that is constantly on call anytime something breaks. This person tells me that he calculated his effective hourly rate with this schedule and came up with a figure of $0.70/hr. $0.70/hr to be a software developer and systems administrator on call 24/7. No thanks, i’ll pass no matter what vague promises about the future you make to me.

Really? Is this the best that we can do? No wonder we have a hard time getting anything done and this space is considered a joke to many outsiders. I don’t think this particular joke is funny at all.

On a related note, I was just recently made aware of a comment that claimed an $800 bid to implement export, which requires BOTH viewer side and server side changes was “not great”. As if this bid to do custom software development on two separate platforms were easy and should pay minimum wage? What?

If you think paying $800 for software implementation is too much, maybe your business should be charging more to free up real money for the software development that it depends on. People’s time is not a charity to be exploited.

Software development charges from companies and independent contractors typically land in the $70 – $120/hr range. In my experience these are fair numbers, and when you pay a much much lower price to try to get the best deal, you end up getting what you pay for, and many times it’ll have to be completely redone later on. When you have to do something two and three times, the cheap rates end up not meaning very much.

So my final message is pretty simple. As we free up more cash to pay more people for their time, expect that the contractors you’re used to paying pennies to may end up finding their way over here. I won’t make any vague promises. I’ll ask them flat out what kind of compensation they think they need to complete a project and pay accordingly. I won’t put anyone on call 24/7 unless they’re being paid real wages. If they go over budget due to a bad estimate, we’ll work to make it right.

I actually like people. I want to see them succeed and be happy. Only when I see happy people working for us and alongside us do I know that we’ve truly succeeded.

My history with InWorldz

My history with InWorldz

[The following was given to the organizers at the InWorldz 6th birthday celebration. I’m pasting it here for those that haven’t been able to read it]

In 2009 I lost my father at the early age of 60. I had a pretty rough job where I was going 12 hrs a day on average on business software. It was mostly web development and database work which wasn’t really my cup of tea. I was in a pretty bad state and I really was looking for a way to dig myself out of the emotional hole I was in.

I found Second Life as an outlet for my feelings. It was great to be able to express myself artistically. I found that I did a decent job at 3d modeling and that my programming experience translated to scripting.

I created a bunch of 3d stuff that I always gave away, and even owned part of a region for a time. I’ve always enjoyed sharing happiness and experiences with others where I could, and I considered renting a full region to try my hand at really reaching out to people who may have been feeling the same pain of loss and letting them know that they were not alone. The biggest problem was that I found was that it was very difficult to justify paying what amounted to a car payment for entertainment.

I did a lot of work in the IBM sandboxes which were a great place to get quiet work done. They were always kept clean and free of drama by PatriciaAnne Daviau a wonderful person who would become a great friend in my real life as well as my virtual life. Patty knew a Scotsgreymouser Janus who in turn knew Elenia Llewellyn.

Elenia Llewellyn and her business partner Legion Heinrichs were looking for a developer to work on server side code for a piece of 3d simulation software called OpenSim. It looked like a good opportunity for me to segway more into the games and visualization side of programming which I had wanted to learn more about anyways.

I agreed to work on the software and I got my own region there to build on. I ended up creating a place that really helped dispense of the emptiness that I felt. To this day “Tranquillity’s Pad” remains mostly untouched. It is my vision of an afterlife where we transcend physical boundaries and can visit far away places as part of a better, more peaceful existence.

Of course, InWorldz work wasn’t all fun and games. Just about every script I brought in crashed the simulator and I ended up having to do massive amounts of work just to try to get the engine stable for any real work. Eventually, Phlox was born out of my frustration and the frustration of our (unexpected) influx of customers. Phlox completely replaced the legacy script engine and runtime, and brought the first of many giant leaps in stability to InWorldz. I was brought on as a founder and owner of the company, and the rest is history.

I and the rest of the InWorldz staff continue working hard to provide an experience that can be as transformational to others as it was for me. InWorldz is about self discovery, and I hope it can continue to provide others with a refuge when they don’t know where to turn.

Finding the height of any point on a 2d plane perturbed by a sine wave

AKA: Making water waves

I wanted to make a special effect for some software that I’m writing. It involved simulating waves in water inside the vertex and fragment shader in an OpenGL program.

While I found a lot of posts on the subject, none of them worked from the ground up to explain what was going on with the math (mostly trig) in the calculations. Since I work a lot better when I can see examples visually, I wanted to share my thoughts while I was going about the process of understanding what I was reading in my head.

There wasn’t a whole lot as far as code because much of it is hidden behind vector operations like the dot product, but really digging down I found the methods interesting nonetheless. I hope my brain dump can make other people excited about some of the stuff behind 3d.


There are many ways to look at a sine wave. Most people know it as the uppity downity regular pattern associated with waves of all kinds. I like to look at it depicted as a continuous revolution around a circle being drawn by a marker.

Sine wave amplitude being drawn from the Y component of a rotation around a circle.

One sine wave is completed each time we go all the way around the circle. This means that as we input between 0 and 2*PI radians into the sin() function, we complete one whole wave. Once we go over 2*PI radians, the function starts over. Another way of saying this is that the domain of sine is between -1 and 1: No matter what goes in, something between those numbers comes out.

Lets say we want to create one of these waves “traveling” in some two dimensional direction given a change in time. We then want to render that traveling wave on a computer screen. We also want to be able to control the frequency of the wave. How many peaks and troughs we’ll see given a single span of 1 unit.

As shown above, we know to complete a full wave we need to input 2pi to the sin function. Therefore, we should multiply our desired frequency by 2pi.

float frequency = 2*pi * desiredFrequency;

You can think of this mutiplication as creating a situation where we pass in the same X value of 2, but more waves have been generated by that point.

Screen Shot 2015-03-06 at 6.34.41 PM

Multiplying by frequency

Since the sine wave is going in a certain direction, the last thing we need to know to draw a point on our plane is how much the position of the point is contributing to amount traveled along the wave.

We can figure this out by obtaining the dot product between the direction the sine wave is traveling, and the location of the point which is itself a vector rooted at the origin.

Screen Shot 2015-03-06 at 6.43.41 PM


Remember that the dot product tells us the cosine of the angle between point vector from the origin and the wave direction. Cosine according to SOH CAH TOA is the ratio of the adjacent side of the triangle vs the hypotenuse. This is exactly the ratio we need to determine how much of the magnitude of the point contributes itself in the direction of the wave.

Putting this together in code, we get:

float theta = dot(waveDirection, vec2(x, y));

Theta is then how much x, y point/vector has moved us along the wave.

Now, finally, we can calculate the height of the wave at the given X,Y point.

//inputs are:
//vec2 waveDirection: The direction the wave is traveling
//float freq: The number of complete waves per unit distance
//float amplitudeMul: The amplitude multiplier for this wave
//float time: input time to create wave movement
//float x: X coordinate of the point we're drawing
//float y: Y coordinate of the point we're drawing

float wave(vec2 waveDirection, float freq, float amplitudeMul, 
    float time, float x, float y) {

    float frequency = 2*pi * freq;
    float theta = dot(waveDirection, vec2(x, y));
    return amplitudeMul * sin(theta * frequency + time);

Happy waves!

CentOS 7/RHEL 7 and docker containers on boot

I recently had to work with a CentOS 7 system and a few docker containers. These containers were bound to different IP addresses on the server NIC and it was critical that they be available after a server reboot. CentOS 7 comes with docker 1.3.2 built into the base installation. This version supports the –restart switch as follows:

–restart=”” Restart policy to apply when a container exits (no, on-failure[:max-retry], always)

Setting the containers to restart=always, rebooting the box, and running docker ps, I discovered that the docker containers were not starting with the OS. After going through a few combinations of commands unsuccessfully I decided to take a look at the message log for the system and I noticed something odd:

Received an unexpected error during port allocation: Error starting userland proxy: listen … cannot assign requested address

Taking a closer look at the logs I noted that this was often followed later by something like:

kernel: e1000e: enp0s25 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: Rx/Tx

It looks like the docker container engine was starting before the NIC was actually up. It was then trying to do its magic and create the bridge interface but the NIC with the given IP address wasn’t ready yet.

To solve this issue I looked around and found an article about systemd services specifically dealing with depending on the network interface to be up. Inside the systemd unit for docker, the service was set to depend on


Description=Docker Application Container Engine
Documentation= docker.socket

Additionally, /usr/lib/systemd/system/docker.socket was not set to depend on at all. To get everything rebooting with the system, I did the following:

  • Added to the Requires and Wants of both the docker.service unit and the docker.socket unit.
  • Ran
    systemctl enable NetworkManager-wait-online.service

    to stop boot processing until the network is available.

Both of these steps may not have been needed and you should play around with the configuration to see what works for you. My final .service and socket files were as follows:
[ddaeschler@docker ~]$ cat /usr/lib/systemd/system/docker.socket

Description=Docker Socket for the API



[ddaeschler@docker ~]$ cat /usr/lib/systemd/system/docker.service

Description=Docker Application Container Engine
Documentation= docker.socket

ExecStart=/usr/bin/docker -d $OPTIONS $DOCKER_STORAGE_OPTIONS


After making these changes, the containers start up normally at boot. I hope this can help someone else running into this issue.

Xcode handy key bindings

This is a tiny postlet on some key combinations I have found handy while using Xcode. I’ll update this as I run into more invaluable shortcuts.

  • shift + cmd + o – Quickly open file
  • shift + cmd + j – Reveal in project navigator
  • ctrl + cmd + up / down – Jump to next counterpart (ex. .h/.cpp file)

A ton more are listed here:

Relay for Life on InShape!

Relay for Life on InShape!

On November 1st 2014, something profound is going to happen, and I’m super excited to be a part of it.

For the first time, avatars and the people behind them are going to run the American Cancer Society Relay for Life, for real, virtually.

What in the world does that mean?

RFL_Snapshot_001Using the InWorldz InShape system, beta testers from the virtual world will get together on a virtual track that has been designed by Relay For Life of InWorldz volunteers. They will log into the virtual world, and by using acceleration data supplied by the cell phone in their pockets, they will transmit walking, biking, and running forces from their real life into their avatar. That data will be used to move them through the Relay for Life track.

From exercise bikes, to rowers, to treadmills, the harder they run in real life, the faster their avatar will go. Their dedication will show through form the real to the virtual. Their avatar will become an extension of the good we all want to see in the world.

Business stamina

We will gather from around the world. We will run the same track for the same great cause, and we will help to give hope where hope is needed.

This is what virtual worlds are all about. Fighting for good causes and using virtual worlds technology to augment the human experience. If you are an InShape beta tester, please join me on November 1st to help kick cancer right in the backside. Let’s get together and sweat to draw attention to a disease that affects us all, and needs to be eradicated before it can claim any more lives.

Run for life.

(Watch the livestream here starting at 9 AM PDT)