End of the Line

Australis has had a long history of tangles with fishing gear. But on New Years Day, 2015 her EPIRB was set off one last time.

In 2004, while anchored off San Quintin, Mexico, a fishing trawler passed by in the night, cutting the anchor line. Australis (Wind Rose at the time) was beached before her crew could react. She was salvaged and refurbished. In 2008 on her maiden international voyage as Australis, she got tangled in lobster traps while anchoring in a bay off Cedros Island in preparation for a storm. The storm shifted the winds to onshore and when we went to crank the engine discovered the fouled prop. Her anchors began to drag as fishermen attempted to help by pulling on her stern. The anchor shackle snapped and we were able to retrieve the other anchor and a trawler was able to bring her to safe harbor on Cedros.

In 2009, halfway around the world while heading into port in Indonesia for customs check-in at sunrise, Australis brought with her a 2 mile long long-line, including 6 inch hooks and buoys. Fortunately we discovered the stow-away before one of the many commercial vessels tangled in it along with us.

Australis lost the last of her 9 lives in Indonesia when Shaun was attempting to make port in Indonesia during a storm...Australian Shipwrecked in Indonesia

Pyopencl (GPU) vs Numpy (CPU) Performance Comparison

While taking the Udacity parallel computing class I decided to compare performance between CPU (serial) and GPU (parallel) implementation. So I ran a simple kernel, `a * a + b * b`, for an array of 32-bit random floats between 0 and 1 using pyopencl and numpy. I wanted to see if it was worthwhile to take my GPU BTC mining rig offline periodically to do scientific computing. But even my a bottom-of-the-barrel 64-bit AMD Sempron 145 (2.8 GHz, 5600 bogo-MIPS) processor with 8GB DRAM can beat a Radeon 7750 GPU for all but the largest arrays. And the best-possible throughput gain on the GPU is about 4x.

So numpy must be well-optimized. I tried optimizing my GPU kernel by minimizing array index lookups, etc, but nothing I came up with made a significant difference for this simple kernel. Both the CPU and GPU gave exactly the same answer, so that's nice.

Here's the output with execution times...

GPU execution time: 0.0115399
CPU execution time: 2.7895e-05
CPU/GPU speed ratio for 10^0 kernel executions: 0.241726% 
Difference between the 2 answers:
0.0
GPU execution time: 0.0115771
CPU execution time: 2.19345e-05
CPU/GPU speed ratio for 10^1 kernel executions: 0.189464% 
Difference between the 2 answers:
0.0
GPU execution time: 0.0116088
CPU execution time: 2.19345e-05
CPU/GPU speed ratio for 10^2 kernel executions: 0.188947% 
Difference between the 2 answers:
0.0
GPU execution time: 0.0115681
CPU execution time: 2.59876e-05
CPU/GPU speed ratio for 10^3 kernel executions: 0.22465% 
Difference between the 2 answers:
0.0
GPU execution time: 0.011663
CPU execution time: 7.70092e-05
CPU/GPU speed ratio for 10^4 kernel executions: 0.660289% 
Difference between the 2 answers:
0.0
GPU execution time: 0.023535
CPU execution time: 0.000612974
CPU/GPU speed ratio for 10^5 kernel executions: 2.60452% 
Difference between the 2 answers:
0.0
GPU execution time: 0.0234549
CPU execution time: 0.0182121
CPU/GPU speed ratio for 10^6 kernel executions: 77.6472% 
Difference between the 2 answers:
0.0
GPU execution time: 0.0668991
CPU execution time: 0.240016
CPU/GPU speed ratio for 10^7 kernel executions: 358.773% 
Difference between the 2 answers:
0.0
GPU execution time: 0.567215
CPU execution time: 2.24371
CPU/GPU speed ratio for 10^8 kernel executions: 395.566% 
Difference between the 2 answers:
0.0





With cgminer running at -I 9 on all the GPUs, the speed advantage for a GPU doesn't budge, significantly.  So pyopencl is pretty effective at interrupting cgminer and prioritizing its threads.




GPU execution time: 0.179582
CPU execution time: 2.7895e-05
CPU/GPU speed ratio for 10^0 kernel executions: 0.0155333% 
Difference between the 2 answers:
0.0
GPU execution time: 0.263615
CPU execution time: 2.31266e-05
CPU/GPU speed ratio for 10^1 kernel executions: 0.00877287% 
Difference between the 2 answers:
0.0
GPU execution time: 0.263666
CPU execution time: 2.40803e-05
CPU/GPU speed ratio for 10^2 kernel executions: 0.00913287% 
Difference between the 2 answers:
0.0
GPU execution time: 0.011616
CPU execution time: 2.81334e-05
CPU/GPU speed ratio for 10^3 kernel executions: 0.242195% 
Difference between the 2 answers:
0.0
GPU execution time: 0.0116951
CPU execution time: 7.60555e-05
CPU/GPU speed ratio for 10^4 kernel executions: 0.650317% 
Difference between the 2 answers:
0.0
GPU execution time: 0.023536
CPU execution time: 0.000617981
CPU/GPU speed ratio for 10^5 kernel executions: 2.62569% 
Difference between the 2 answers:
0.0
GPU execution time: 0.0236619
CPU execution time: 0.0189419
CPU/GPU speed ratio for 10^6 kernel executions: 80.0524% 
Difference between the 2 answers:
0.0
GPU execution time: 0.0630081
CPU execution time: 0.230431
CPU/GPU speed ratio for 10^7 kernel executions: 365.717% 
Difference between the 2 answers:
0.0
GPU execution time: 0.82972
CPU execution time: 2.4491
CPU/GPU speed ratio for 10^8 kernel executions: 295.172% 
Difference between the 2 answers:
0.0
Installation was a bit tricky. You have to make sure setuptools is overriden by distribute. But Ubuntu 12.04 makes this easy. Thanks to kermit666 on SO for this simple approach to getting virtualenv wrapper and numpy up and running quickly on a fresh Ubuntu install.


#!/usr/bin/env sh
sudo apt-get install python-pip python-dev
sudo pip install virtualenv virtualenvwrapper

echo 'export PROJECT_HOME="$HOME/src"' >> $HOME/.bashrc
echo 'export WORKON_HOME="$HOME/.virtualenvs"' >> $HOME/.bashrc
echo 'source /usr/local/bin/virtualenvwrapper.sh' >> $HOME/.bashrc

sudo apt-get install -y gfortran g++
# sudo apt-get remove -y --purge python-setuptools

# start a new virtalenv project
mkproject parallel
pip install --upgrade distribute
pip install mako numpy pyopencl

Here's the python code that ran the kernel and measured execution time. It's based on the official pyopencl example ...

import pyopencl as cl
import numpy
import numpy.linalg as la
import time

for M in range(0, 8):
    N = 10**M * 1000
    a = numpy.random.rand(N).astype(numpy.float32)
    b = numpy.random.rand(N).astype(numpy.float32)

    ctx = cl.create_some_context()
    queue = cl.CommandQueue(ctx)

    mf = cl.mem_flags
    a_buf = cl.Buffer(ctx, mf.READ_ONLY | mf.COPY_HOST_PTR, hostbuf=a)
    b_buf = cl.Buffer(ctx, mf.READ_ONLY | mf.COPY_HOST_PTR, hostbuf=b)
    dest_buf = cl.Buffer(ctx, mf.WRITE_ONLY, b.nbytes)

    prg = cl.Program(ctx, """
        __kernel void sum(__global const float *a,
        __global const float *b, __global float *c)
        {
          float a2 = a[gid];
          float b2 = b[gid];
          c[gid] = a2 * a2 + b2 * b2;
        }
        """).build()

    prg.sum(queue, a.shape, None, a_buf, b_buf, dest_buf)

    gpu_ans = numpy.empty_like(a)
    gpu_t0 = time.time()
    cl.enqueue_copy(queue, gpu_ans, dest_buf)
    gpu_t = time.time() - gpu_t0
    print 'GPU execution time: %g' % gpu_t

    cpu_ans = numpy.empty_like(a)
    cpu_t0 = time.time()
    cpu_ans = a * a + b * b
    cpu_t = time.time() - cpu_t0
    print 'CPU execution time: %g' % cpu_t

    print 'CPU/GPU difference in speed for %d additions: %g%% ' % (N, 200.0 * cpu_t / (gpu_t + cpu_t))

    print 'Difference between the 2 answers:'
    print la.norm(cpu_ans - gpu_ans)

Bitcoin's Easter Sunday

I think I've finally figured out Bitcoin (BTC). I've been looking for the attractor of the dynamic system that is the Bitcoin network as it has risen to ever irrationally exuberant levels. I assumed that it was like any other economic system, that it would be modelable in terms of scarcity and consumer psychology (perception of value) or social evolutionary dynamics. But maybe it isn't. Maybe it's driven by a new kind of mechanical growth, the growth of the brainpower in a thinking machine. Maybe Kurzweil's exponential thinking is right. Only the singularity isn't coming to us meat-space humans, it's coming to machines first. I now think that the value of a Bitcoin seems will increase in direct proportion to the energy efficiency that the Bitcoin "machine" (network) thinks with about a very specific problem. A machine that thinks about nothing but finding the "secret" to each and every SHA256 hash "problem" that ripples across the network. It's no coincidence that the advent of ASICS in bitcoin mining has spawned another burst of speculation in Bitcoin. I think that growth in value would have happened with or without human speculation and greed. The speculation may go away, and the pendulum of human opinion may swing the other way soon, but the the long term metoeric rise of Bitcoin is here to stay. And we've crossed an inflection point in confidence and value that will never evaporate... because it no longer depends on the scammers and speculators that have yanked the price around for the 4 short years of the Bitcoin "machine"'s life. Sure, we slow humans write the software (Bitcoin miners) and build the specialized machines (ASIC racks) that act as the neurons in this brain. But it really seems to grow organically, more like an epidemic or organism. Maybe it isn't a fad, a bubble. Maybe we're witnessing an inflection point in economics itself. The new currency is thought, and machines will be the arbiters of that currency, thinking faster and better than any Barron, banker, or entrepreneur could. Bitcoin recently crossed the $1B capitalization level (total "money" supply). This is more than 1% of the total supply of all the money in the United States--all the government debt, all the bad home loan debts held by your local savings and loan, all the "paper" money (derivatives contracts) printed by the huge Wall Street banks. And this Bitcion capitalization is 1% of all the value and all the promises of value held in the US--and it's contained in a currency that you've never heard of, and that no government or corporation controls or processes in any way. However, one company, Avalon, seems to have figured out a clever way to make money off of the Bitcoin craze, seemingly without conning it's customers or contributing to the irrational exuberance. They're making money the same way that thousands of geeky innovators have been making money off of Bitcoin for the past 4 years... by mining with ever-increasingly powerful and innovative machines, or selling that equipment to others. Only Avalon is clever about trickling out the supply of their hardware (and probably using the backlog of equipment to mine with themselves). Just fast enough to keep leapfrogging their competitors (FPGAs and GPUs and other less legitimate ASIC manufacturers) without flooding the market with valueless processing power. They have skin in the game, so they want the rise in value of Bitcoin to continue, or they cease being profitable. I'm going to spend the day exploring the "strange attractor" that governs the price and spread of Bitcoin. I'll let you know how it goes.

The 2nd Great Depression is Here -- and that's Good News for the Smart

I love this FT series on the economics and politics of deflation and recession.

http://ftalphaville.ft.com/blog/2012/06/08/1030801/the-end-of-artificial-scarcity/

Near as I can tell FT is saying "don't worry, be happy." Fortunately smart people worry and we are a social animal.  Pack social dynmics are ushering in an era when national governments are no longer relevant to our material happiness. Barring WWIII, smart people will continue to band together into clubs, clans, coops. Just wander around Portland for a day. They barter within a web of trust, set up insurance pools, loan each other money--even mint their own fiat currencies (bitcoin) or fund grand scientific and space adventures. These pockets of stability and hope will thrive while the rest of us decide which "Like" buttons to click.

Gardening

As is Vanuatu tradition, when we rowed ashore at a small island north of Port Villa, the first man to greet us, 'Paul', became our host and guide for the duration of our stay. He regularly offered us valuable cocoa pods as a snack and other delicacies from both his own farm and his neighbors'. He looked quizzically at us whenever we'd ask him (in broken pidgin or English) whether it was all right to take food from a neighbor. They have a liberal sense of property rights in Vanuatu. Though Paul worked several hours a day to keep his 'garden' productive, he also helped himself to his neighbors' produce. Perhaps his 'hosting' duties were his justification. We helped him plant taro root in this plot of land that he'd slashed and burned. He lit several piles of limbs that we helped him gather and they smoldered all day while we worked. It's sad to see the amount of land destroyed to maintain a single family for a year. But the land is so fertile and plentiful that there is not much sense of conservation in Vanuatu.

Irregularly Sampled Time-Series

As I was researching options for analysis of irregularly sampled time-series data for the bitcrawl project I started to wonder whether it even made sense to think of the data in terms of a time series. All the articles I read come from the viewpoint of analyzing a physical system, looking for things like the power spectral density or wavelet functions, which can help pull out the underlying character of a linear dynamic system hidden beneath noise. But there is no such system in the financial markets. When traders talk about high frequency trading with bandwidths exceeding 1 MHz, that's when you realize that the dynamics are those of computer algorithms and complex systems, not physical systems. It's like watching the lottery ping pong balls rattle around in a glass sphere. It's totally meaningless. Thermodynamics, with bandwidths in millihertz is the only reasonable physical system for modeling that kind of complexity. I don't want to compete with the ping pong ball catchers that have to build ever faster data centers and fiber networks to keep up with the speeding bullets. Human social dynamics take over in the millihertz range. And when you start talking hours and days or even months between trades (as for me) you need a more all-encompassing metric that reaches into the really low frequencies for which you have almost no data. It's like my Low Frequency Impedance measurement algorithm invented 20 years ago--you can extrapolate a single point on a spectrogram from as little as 1/3 of a sinusoidal period.

But markets aren't physical systems that can be probed with pure sinusoid inputs. Instead, it seems to me, your best bet is to think in terms of volumetric spans rather than time spans. Or even better, just in terms of transaction counts. What matters is how far the price moved between trades, not between days, hours or microseconds. How many different people or groups of people or computer algorithms decided to adjust their price and by how much? That's a fair gage of the "temperature" or "pressure" of the thermodynamics of the market. Of course with electronic exchanges facilitating HFT, these independent actors are getting parsed into the tiniest little chunks. So volumetric measures may be even better. That helps get over the fractal nature of the markets. In the end, even a fractal has a volume, as least it's projection in 3-space does. Hopefully the same concept makes sense for the markets, because that's the path we're crawling down with bitcrawl.

But time is money, so eventually we'll have to do the conversion back to real time, based on some average trade frequency or volume rate for a given instrument. But my guess is we'll discover a lot of hidden dynamics lurking in volumetric and transaction-count space. I hope I can find a market to give me this level of detail.Bitfloor is all I've got right now, without succumbing to the price-gouging of Bloomberg or other financial services, or public exchanges.

Towing?

It took a bit of staring to figure out why this ship was doggedly chasing it's bow. There is almost no freeboard near the center of these ubiquitous Indonesian ferry and cargo ships. The low freeboard makes it easy to board without a gangway or boarding ramp when the ship pulls along side a floating dock or another ship--which is surprisingly common. These ships are often docked 3 or 4 abreast, making for a nice raft party, I'd imagine. But once they get underway, the deck would surely collect a lot of water. Maybe that's why the bilge pumps seem to be constantly running on these boats. It scares me when I see them overloaded with diesel drums or human cargo. But I imagine that if you pushed her at just the right speed, her bow wave trough would line up perfectly with this low gap in the middle, ensuring a dry deck and bilge--as long as no waves assaulted from the side.