Circular Dependencies in the real world

In programming there is a problem known as circular dependency, which you know is real because it has its own Wikipedia page and everything.  In summary the problem is that Class A needs to know about Class B, but Class B needs to know about Class A.  The problem arises because the compiler has to pick one to start with.

As an analogy let’s take Bob and Alice who have been married for many years and won’t go anywhere without each other.  Strolling through the park they find the bridge across a stream is out but they are in luck, someone with a two seater swan boat is willing to offer rides.  Since it is his boat (and no one is going to question why in the world he owns one personally) he must remain in it at all times.

Now since Bob and Alice, or Alice and Bob if you prefer, won’t go anywhere alone here is the problem.  The Cygnus captain only has room for one.  He points to Alice and says “I’ll take you across first” to which she responds “But Bob won’t be on the other side when I get there so I can’t.”  He turns to Bob and says ” if you go first then you will be on the other size and therefore she can cross right after.”  Bob replies “That sounds great but only if Alice meets me across the river when I get there.”  Exasperated, holding his face in his hands, the captain doesn’t see the iceberg coming downstream which proceeds to sink his swan.  Luckily the river is only a few feet deep but still he winds up wet and boat-less.

He hit an impass, there is no way to get one to cross without the other being there first.  Despite The Substitute teaching otherwise, sometimes the best solution is not to throw a boombox out a second story window.  Outside of programming this is known as the chicken or the egg problem.  Clearly one came first but there is no clear cut answer as to which is correct (the egg is the right answer btw).

Why am I talking about eggs?

Other than the fact that they are delicious, this problem comes up ALL THE TIME.  Just today I was listening to a cross discipline engineering project where the hardware people said they couldn’t finish without the software (true) but the software people couldn’t get started without the hardware it runs on (also true).  Think back and it shouldn’t take too long to come up with similar situations in your life.  Businesses (especially tech support) thrive on this.

“To recover your pin number, please type in your pin on the phone keypad now.”  Most cases aren’t this bad but stuff that stupid does come up now and again.  Sadly the people involved are likely alive and employed.

How do we fix this?

You’ve convinced me I hear you saying, now how do I solve these seemingly insurmountable cycles?  First I say: Get out of my house!  Secondly we have to take a look and see if we really need one before the other, if not hooray the problem just fixed itself.  The key to break the cycle is to do anything that breaks the cycle.  In the world of making things that depend on other things this would be a prototype.

Wait, did you say hardware Prototype?

Not exactly, you filthy word injector, but yes that is what was implied.  Certain situations make prototypes more than helpful, they become essential.  That situation would be any time you don’t know how to do something.  Since any interesting project will have at least some unknown aspect this covers 99.9999+% of all products.  People thought all vacuums were the same until James Dyson got tired of clogged filters and did something about it.

The way to  break the cycle is to build something, anything that gets you closer to the goal.  IT WILL BE WRONG  and that is okay.  Failing when you absolutely, 100% upfront intend to fail isn’t bad, it’s only bad if you don’t learn from the experience.

Say you are building a widget; I’ll wait until you finish speaking to continue.

This widget is revolutionary and will sell like hotcakes back when people used to buy a lot of hotcakes.  The only problem is like many products it requires hardware and software.  Damn, now we are really screwed because apparently programmers and engineers (not software engineers) have a fundamental mental disconnect in how they approach problems.

walrus and carpenter

The Engineer and the Programmer, one of which is about to get nailed in the bean with a hammer

The Hardware Engineer Says:

I have to design this perfectly up front, it is too expensive otherwise.   If I miss one detail it will take months to get the manufacturer to retool the assembly line, that will costs tons of money, hell it may kill the entire project.  Better take as much time as it takes to get it right the first time.  Wish that lazy Programmer would tell me the specifics he needs, then I could spec just enough power to make it work.

The Programmer Says:

I have never done anything like this before, there is no way I can get it right the first try.  Would love to make a quick prototype that supports the most important features of the product but the hardware to run it doesn’t exist yet.  What the hell is wrong with that Engineer wasting time on crap that will clearly be wrong the first time, if not multiple times?

Who’s right?

As one of my professor’s says “it depends,” which is sadly almost always the correct answer for a complex situation.  Both the Engineer and Programmer raise good points.  It does cost money to make mistakes but it is hubris to assume that by working really really hard you can get it perfect the first time.  In the computer world contradictions this strong cause your compiler to lock up, machine to blue screen (even on Linux), and the cpu to burn itself to ash to avoid the agony.  The real world, however, thrives on this shit.  Those familiar with quantum weirdness know that Schrödinger’s cat is both alive and dead at the same time, at least until you open the box.  Chickens and eggs don’t care which one came first, they just keep doing whatever it is they do (being delicious).

Pictured: deliciousness

That wasn’t an answer at all!

Damn, you caught me.  So here is my opinion on how you solve the problem.  In building our widget you get the engineer to create a really cheap prototype that does the absolute essentials probably right.  But it is cheap and fast.  He then gives this to the programmer who can start prototyping while the engineer works on fixing obvious and not so obvious design flaws.  You know, the kind that make you slap your head and say “wow, wish I had known about that back in the design phase.”  The really important part is that they share feedback throughout the product life cycle.  If the engineer flings a circuit board over the wall then bit later a CD comes bouncing back it absolutely won’t work.  Close feedback is the key to making this work.  Note: you can have good feedback without being able to physically talk face to face, these days this thing called the internet solves those kinds of problems.  In the software world this form of management is called Agile Development.  In the engineering world it is called Blasphemy.

So we are starting out this supposed year long project.  Team A is using the old standby methods to build, Team B is trying out new, more radical thinking.  Suppose the deadline is absolute.  There is no reasoning with it, no extending it, no nothing.  Here is how these projects work out often enough.  Numbers may or may not be accurate, but they prove my point.

DateTraditional EngineerTraditional ProgrammerAgile EngineerAgile Programmer
Month 1Got lots of research to do!Lots of research to do!Finishing touches built on first prototype after week one brainstorming session.Been working with engineer to get prototype working.
Month 3Moved on to designing boards and actual engineering.Toyed around with a few ideas but not really sure what the hardware will be so can't write anything.Got lots of feedback from first prototype, started on next, more polished one.Got a pretty good idea where the hard parts are going to be, creating software prototypes of tough stuff.
Month 6 (of 12)Design 90% done but still a few kinks to work out.Playing solitaire as there is no real work to be done until have some device to test out.Last prototype nailed it, now polishing design and adding optional features.Testing of required features mostly done, minor bugs remain.
Month 890% done.Trying to look busy.Final polish being added.Minor features being coded.
Month 10Done with board, passed off to programmers.Coding franticly.More final polish, working with marketing to improve product launch.Optimizing code and working on documentation.
Month 11Waiting on feedback from lazy programmers.Code harder than first expected, putting in extra hours.Started work on new project while waiting for launch.Final cleanup and ready for release.
Month 12Yelling at programmers to get done.FUUUUUUUUUUCK!Enjoying work on second product while success of first happens.Started on code for second product, not burned out from first project.

Yes this is an extreme exaggeration, but it’s my blog so tough crap.  And many real world products move along exactly the same way as the first team.

Summary

If you want to break the chain of impossible dependencies you have to do something!  Planning gets you no closer to shipping when you cover the same ground over and over and over again.  Take a chance and build something that might work, or might not.  As long as you aren’t making life critical products (like a pacemaker) this is not only a good way to go, but in my opinion it is the best way.  Look for future posts on how waterfall design methods (that all engineers use) are crap and agile hardware development.

Confessions, consessions and depression

I give up.  After years of fighting everyone in my department I give up.  They win.  What am I talking about?  Which operating system to write code on.  At UAF you start of using Visual Studio, much to the dismay of many students.  Creating small single file projects with MSVS is like killing flies with a bazooka; it may get the job done but there is a lot of collateral damage.  And I am tired of picking up the pieces.

All the tools available, particularly the  ones related to my thesis, either “work” on Windows, theoretically work on Windows, or openly flame Windows.

So starting this weekend I am going to install Linux (Ubuntu most likely) and refresh myself in the ways of gcc.  Sigh.

Dreamhost Promotional Code

To celebrate actually still being alive, albeit less active on the web, I’ve created a promotional code for my all time favorite web host Dreamhost.  Aside from having excellent customer support in my experience, they have ridiculous goodies to play with.  My all time favorite would of course be Subversion.  But you don’t care about any of that, let’s get to what you want.

The code: COLLEGECODEDSCNT

The benefits: 10% off of ANY package length of time you sign up for.  If you buy a year save $6 on top of the crazy regular Dreamhost promos.  If you buy a decade save over 70 bucks!  This is on top of the free domain name, excellent service and all the other good things that come with using Dreamhost.

So what are you waiting for?  Sign up for a new account now!  Who knows how long this snow influenced dementia will last.

Software Manifesto

manifesto

Today I stumbled upon an interesting manifesto.  No not that one, the software one.  A call to arms for all professional software writers, as opposed to code monkeys.  Link for those who wish to skip my blabbering.  It isn’t very long, which is good because I don’t sign stuff I don’t read.  It is and I quote:

As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:
Not only working software,
but also well-crafted software
Not only responding to change,
but also steadily adding value
Not only individuals and interactions,
but also a community of professionals
Not only customer collaboration,
but also productive partnerships

That is, in pursuit of the items on the left we have found the items on the right to be indispensable.

A noble and worthy goal, I signed.  I would also encourage anyone else who holds these to be true to sign as well.

DVD Ripping the Windows Way

I finally decided to digitize my dvd collection.  Two things prompted this, 1 the rack is running out of room (see below) and 2 they are all at my parents house.

dvdrack

This prompted research as how to break the annoying dvd decryption and obtain personal use backups of dvd’s I actually legally purchased (as opposed to many of my other collections).   Googling quickly showed a few programs that work and I find useful.  The first being the zombie program DVD Shrink.

While DVD Shrink is no longer being worked on,  you can still find it floating around the net.  The most recent version is 3.2.0.15, note it is freeware so avoid the sites that charge for it.  Or just download it here.

The next piece of software used is called Handbrake, which takes dvd formatted data and converts it to normal video files such as mp4, avi or ogm.  It is open source and therefore free.  The thing I am liking most about it is the ability to queue up multiple conversions and let it run overnight, which is important as it takes around 1.5 hours on my gaming machine per movie.

The two afore mentioned pieces of software work most of the time but since DVD Shrink is a few years old it can’t break some of the newest encryption methods (damn you Disney).  This is annoying because Wall-E is an awesome movie.  So to get around this I found AnyDVD HD.  It tears out the encryption on all dvd’s, the HD version even works on blue-ray and HD discs which is awesome.  The only not awesome part is the price, 79 euros = $104 real dollars.  There is no way I was dropping that kind of money on software for a single week long ripping party, luckily they have a 3 week trial version.  The only downside is it forgets your settings when it shuts down, which only matters if you turn off your computer (I don’t).

With my newly assembled arsenel of software tools I began tearing into the rack of disc’s like a rabit badger, or at least a wounded chipmunk.  Here is my method.

DVD Shrink makes fine backups but as they are huge (4+ gigs each, some over 8 ) I decided to convert them to the more portable and popular avi format.  Max quality avi’s of these work out to be around 1.6 gigs tops (Lord of the Rings), pretty close to 1.4 gigs on average though.  If space is an issue then they can be shrunk later on.  Let’s start making the MPAA cry.

First you insert the dvd in your machine and launch DVD Shrink.  Then choose open disk.  If you have it AnyDVD should be launched first as it intercepts dvds before you open them in DVD Shrink.  Closing or modifying AnyDVD settings while ripping with DVD Shrink causes it to lose connection to the disk and you have to start over.  When the fox icon is pink it is thinking, when it is red it is ready.

rip1

Wait while it analyzes the movie, this takes about 2 minutes.  Usually mine start off at a pretty slow speed, like 2,000 kbps then work up to 9,000 max.  The encoding phase does this as well.

rip2

If you want to back up the entire disk, menus extras etc, then just choose backup after the dvd is analyzed and skip the next step.  If like me you are only interested in the feature film go to re-author and drag the title from the “Main Movie” area over to the empty area on the left.  Now when you choose backup it will only save that one, cutting down on important disk space.  I have a 1 terabyte drive as the buffer for this but 7 gigs adds up pretty quickly.

rip3

Once you have the tracks to rip, let ‘er rip.  It takes anywhere from 7 minutes to an hour, depending on how much there is to work with and if you chose any other fancier options.  Generally get less than 10 minutes though.  Again it starts off slow and works up to a decent speed.

rip5

This is the message you are looking for, the a-ok to move on.  Congratulations, you are now out a large chunk of disk space.  If you are ripping multiple disks at once make you to switch back to “Full Disc” mode as otherwise it pops up an annoying message and beeps at you.  Also you can’t open a new disc until you change to full disc mode.  Note the discrepency in times between the previous picture and this one.

rip7

Next we can start converting the big honking mess of dot vob files into a usable format.   Start up Handbrake and choose Source in the upper left corner, then select the DVD / VIDEO_TS Folder.  Find the folder you want to convert on your hard drive and select it.  Sometimes you have to choose the VIDEO_TS folder itself, the immediate parent one won’t work.

rip8

Then you choose the file name, type and all the other settings I don’t mess with to create the actual file you wanted in the first place.  Normal has worked in all cases, I just change the file type.  Make sure to look at the title drop down and check that the file length is appropriate (seen here), some of them have multiple ones to screw you up.  Annoying to waste 2 hours to find out you chose the wrong chapter to burn.

rip9

Choose where you want to save the file and what to call it.  Also the file type, I went with avi as it is widely supported and ogm didn’t work on my machine (will figure that one out later).

rip11

Go over all the settings to make sure it looks good then start it going.  The start button makes it go right now, alternatively you can hit the “Add to Queue” and make a big list of them for your computer to work on while you sleep.  It is very cpu intensive so best to let it run alone on your machine if at all possible.  When you hit Start you get a friendly CMD.exe window that has an eta on it.  Don’t close this window.  It works in two stages, the first is encoding which is the fast phase.

rip12

Then there more encoding which eventually writes it to disk.  Takes over an hour for my machine per dvd so settle in.  This next picture is just to show what the second phase looks like.

rip13

The next to final stage is to verify the rip actually worked.  Choose your favorite media program (I highly recommend VLC) and load the file, make sure the sound is sync’d right and other such things.  Note the video did work, the screenshot capture method didn’t.

rip14

When you know the video ripped correctly go back and delete the raw dvd folder as it is just a waste of space.  Now you too in around 2 hours can make digital copies of stuff you already owned.  Sad fact of the day: with a fast internet connection it is  quicker to pirate the movies than convert the ones you own.  But piracy is wrong, and backing up dvds is mostly legal.

One final tip, you can run as many copies of DVD Shrink as you have dvd drives.  I crank on two at a time, tried to put in a third pulled from an old computer but my motherboard didn’t have a second ide cable.

rip151

Have fun backing up you movies!