Struggles and Success With Cygwin

While PowerShell is a great thing for Windows management, it still has a long way to go to catch up with the sheer flexibility of Bash and the staggering array of tools centered around manipulating simple streams of text.

This post on StackOverflow contains a few great bits from different perspectives: even from the creator of PowerShell itself.

I just wanted to share some of the problems that came up with my attempts to integrate various open-source tools into our Windows-based development environment. Finally I’ll share some tips/observations that led to an enjoyable environment.

Problems

Permissions

Easily the most frustrating part of Cygwin is when it attempts to imitate POSIX permissions on NTFS volumes by manipulating the ACLs. Frankly, Cygwin does a terrible job at this. It only serves to cause massive headaches.

TURN THIS OFF before you do anything else in Cygwin.

You can avoid permissions issues by setting up your NTFS mount points with the “noacl” option. This is easily accomplished by configuring them in fstab with the “noacl” option, like so:

none            /cygdrive       cygdrive        binary,posix=0,user,noacl       0       0

This line will cause Cygwin to simply leave the default Windows permissions alone. They are good enough in 99% of cases, and I’ve found that they are better than what Cygwin does to permissions (it’s not pretty) in nearly 100% of cases.

Forking

This is a problem that can be harder to work around, due to the fact that Windows doesn’t allow easy forking of processes. Cygwin’s version is very slow (by necessity). The maximum forking performance of Cygwin is about 2% of my comparable Mac OS X machine’s speed when executing Bash commands in a loop. Here’s a useful mini-benchmark to see for yourself:

while true; do date; done | uniq -c

One way around this limitation is to use a programming language like Perl or Ruby for scripts that need to iterate over large numbers of files, etc.. Ruby has a very handy standard library with good file handling utilities.

Rebasing

Sometimes, after an update, Cygwin programs are not linked correctly anymore. This is solved with a rebase. This operation will update your Cygwin binaries/libraries to be able to find the right functions in linked libraries (like the all-important cygwin.dll).

Background Tasks

The cygrunsrv tool lets you install any cygwin program (script or .exe) as a Windows service. This is a godsend, as you can easily get useful servers like rsyncd, git --daemon, simple Ruby web servers, or any other background task up and running with little trouble.

Ruby

Arguably the best part of Ruby is the RubyGems packaging system. Many of the best gems, however, require native extensions to be built. While some gems have pre-built Windows binaries, many do not. Cygwin makes this easy by making GCC and many of the dependent libraries available. I’ve never run into a gem that couldn’t be built under Cygwin with anything more than grabbing the right libraries through the package manager.

Jenkins, Cygwin, Windows 7, and permissions

Cygwin is a godsend for controlling your workflow and putting together the sort of tools that keep devops humming along. And Jenkins is a great freeform build tool for managing pretty much any software CI/build process.

But Cygwin and Windows permissions don’t always play nicely, especially when you add intermediary processes like Jenkins to the mix. My build scripts were copying various folders into the Jenkins workspace, among other things. However, the files and directories ended up having all sorts of nasty permissions issues. Certain users (like the ASP.NET app pool identity) could not read files, and overall things looked pretty weird.

So, Icacls to the rescue! Right?

icacls the-dir /reset /T /C /Q
... <snip 5000 lines of Access Denied errors>

I was getting “Access Denied” errors on every file in the directory, even when I made sure to run cmd.exe as Administrator. Icacls was not working to take ownership either, so I tried Takeown. It worked to take ownership where Icacls did not.

takeown /F the-dir /R
icacls the-dir /reset /T /C /Q

All is well now. I hope this might help someone with the same issues.

Things you can’t do in Visual Studio

Here’s something you can’t do in Visual Studio: be editing code in one half of a split window, and be having a conversation with the language/framework author in another, with only a few keystrokes, and with the full facilities of the editor at your disposal. Thanks, Emacs. You made my day.

Is Zach Holman trolling us?

spark is over 400 points on HN right now.

Initially, the script didn’t work on OS X. Now I’m not sure what it does work on, but it barely works on my OS X terminal, and it definitely doesn’t work in Cygwin.

It takes a comma-separated list, instead of a standard argument list. So you have to use tr "\n" "," instead of just simple substitution or xargs.

It doesn’t even handle the examples in the README. The earthquake data is made up of floating point numbers, and so you not only lose a good deal of information, but the latest script flat-out chokes on it.

And then there’s the speed. Somehow it’s taking over 2 minutes to run the (incorrect) earthquake sparkline.

Now compare that to a 4-line Ruby version that works correctly, handles positive and negative decimal numbers, works the same anywhere that Ruby runs, and runs hundreds of times faster (and this is Ruby here), and holman tells me “Ruby is for chumps”?

Is he trolling?

R.I.P. John McCarthy

The third major computing figure to die in a month! They say famous people die in threes.

Anyway, I owe my career in software to John McCarthy. My first introduction to programming was through the Apple Logo programming environment in elementary school.

Fast forward many years later, after stumbling along in BASIC, PHP, Java, C#, Ruby, and others. I rediscovered Lisp through Scheme (and The Little Schemer) and Clojure. Little did I realize that Logo was actually a Lisp dialect all along, and while it was the first Lisp REPL I would use, it would certainly not be the last! This return to “the programmable programming language” would dramatically reshape the way I think about programming.

Thanks to John McCarthy for his tremendous contributions to the programming world.

Poor Man’s Tuples in Clojure

Hiredman is working on a nice implementation of tuples in Clojure. But, just for fun, I wanted to whip up the absolute minimum viable O(1) tuple possible.

The result is this macro:

(defmacro tuple
  [& xs]
  `(fn [idx#] (case idx# ~@(mapcat list (range (count xs)) xs))))

Try that in Ruby or C# ;)

So, (tuple "foo" 5 :bar) returns a function that takes an index and returns the value at that index in the arguments to tuple, i.e. ((tuple "foo" 5 :bar) 1) returns 5.

The macro compiles to a case form which dispatches in constant time, although it’s not as fast as Hiredman’s implementation, or even Clojure’s native vector.

Getting Started in Rails is Insane

Insane. And not in a good way.

James Johnson posts a “beginner’s” guide to getting a modern Rails environment up and running, in Rails and Web Development - Take the First Step.

(note: I’m not really trying to poke fun at the tutorial here… the point is that the tutorial is actually mostly accurate: please read on)

Don’t people realize that this is crazy? The steps seem to be as follows:

  1. Buy a Mac.

    It’s OK if you don’t have one. They are cheap on Ebay! It will only cost a few hundred dollars to get an old computer to write Ruby like a cool kid. If you want to use Linux, sorry, you’re kind of on your own.

  2. Install Xcode.

    We really just need some of the stuff that comes with Xcode. But we’ll just say we need Xcode for brevity. Just download this 1.6GB package and you’ll be rolling along in no time!

  3. Open the Terminal.

    Don’t worry about what it is or where it resides on your computer. Spotlight will find it for you. And don’t worry about what you’re going to enter into it. Everything will be fine. Trust me.

    We won’t accidentally delete your home folder or anything like that.

  4. Install a package manager.

    What’s that, you ask? Don’t worry about it!

    Just copy and paste this code into your terminal. It will download and evaluate some code that someone posted on a website. It’s fine. Trust me.

    Oh, and if the code that we just blindly downloaded from a site on the Internet asks you any questions? Just hit Enter to say OK.

  5. Now install the Ruby Version Manager.

    “But we just used Ruby to download and evaluate some random code from the Internet”, you say? How astute! Yes, you already have Ruby. But you have the wrong Ruby.

    Trust me. You need a new version of Ruby. And you need extra software to juggle all of these Rubies. It’s fine. Everybody does this now.

    And we’re going to run more code straight from the Internet now. Except, this time we’ll feed it straight to the shell. It’s OK. It’s from Github.

  6. Now just use an editor you don’t know to open a file you didn’t know you had, to add some code you don’t understand.

  7. Now quit the Terminal and re-open it for some reason.

  8. Now install that other Ruby we talked about.

    Yes, yes, I know we’ve already been using Ruby in half of these steps. But like I said, we need a new Ruby. Just copy and paste a few more things. They’ll do stuff. It should be fine.

  9. Update your gems… whatever those are.

  10. Install Rails.

  11. Install Bundler (this is something you don’t need; but you’ll need it).

  12. Now make a Rails app and run some code that will create about a billion different files that you don’t need to understand.

  13. Migrate your database.

    Don’t worry about what a migration is, or where your database came from, or where it is now. It’s fine. Just rest assured that there’s a database somewhere.

  14. Start the server and enjoy!

    You’ve touched compilers, blindly downloaded code, modified your environment, installed a parallel version of Ruby, created a CRUD application and an accompanying database (and data model!), and started an application server, all without understanding a single thing!

    You can use that editor you don’t know how to use to look at files that you don’t know the purpose of.

OK. So this got really sarcastic really fast, and maybe a little nasty. Sorry if I have offended, James. I really commend the effort to get folks into programming. I really do. I want to see people programming, and learning to program. But I also want to see them understanding. And I don’t like to see people starting out on a massive undertaking like this with a “hand-wavy” approach to all of the steps.

But again, my point is not to criticize the intentions here. My point is to criticize the astounding complexity and the piles of “magic” involved in getting up and running with a web framework that’s supposed to be the easy one, out of the bunch.

The only thing I’d change out of all these steps is to recommend an easier text editor. Otherwise it’s pretty much correct about what you have to do to get up and running, and to not be in total isolation from the rest of the Rails world. If you develop a Rails app of any size at all, you will be juggling Ruby versions, gem dependencies, plugins, etc..

And that’s crazy.

After downloading gigabytes of software packages and hundreds of thousands of lines of Ruby code, creating an application and a database, you still haven’t even touched the Ruby language, or any programming concepts.

Stay tuned, and I’m going to see what I can do about this…

Since Everybody Needs to Say Something About Steve Jobs

The passing of Steve Jobs is a really big deal. His contributions to computing made a big impact for me, personally.

We programmed Apple Logo on Apple IIGS computers in the computer lab in 3rd grade. It just “clicked” with me, and nearly 20 years later, I’m still programming (and I have actually come full-circle and picked up Lisp; it turns out Logo is a Lisp). In after-school programs, I tinkered with more advanced (classic) Macs and Hypercard.

After Jobs’ influence had waned, while Apple was cranking out ambiguously-numbered Quadras and LCs and other forgettable machines, I dabbled in programming for DOS and Windows, but nothing stroked my curiosity like those Apple machines. I remained a “computer guy” but nothing much of a programmer.

Fast-forward to high school. I remember the afternoon, waiting with bated breath for OS X 10.0 to be installed on a salvaged Power Mac G3. The UNIX core and NeXTSTEP-descended interface of OS X was the most exciting thing (to me) in computing for a long time. I somehow arranged for a Power Mac G4 as an early graduation present and dove into Xcode, and programming shell scripts, PHP, Ruby, Objective-C, and anything else that I came across.

The promise of being able to make apps that looked like slick Apple apps was motivation enough to learn Objective-C, Cocoa, and Xcode. OS X lit the fire under me to get coding again, and OS X (and attendant hardware; secondary for me, really) have been a constant companion ever since.

So thanks to Steve Jobs for stoking the passion of my hobby and my career.

Flixel with MXMLC

I found the existing tutorials for using Flixel with MXMLC lacking, so I created this guide. I hope it’s useful.

“Patent reform” doesn’t mean what you think it means.

Yesterday, in his speech at a next-generation car battery factory in MI, President Obama asked people to support passage of patent reforms necessary to enable innovation. My first instinct was to cheer this seemingly out-of-the-blue revelation, assuming that the administration had been paying attention and was advocating the kind of patent reform we actually need to ensure unfettered innovation.

I was wrong.

What Obama actually meant was that we should support passage of the Leahy-Smith America Invents Act. This joke of a reform has already passed the House and is on to the Senate.

Others have summarized the problems with this bill better than I can, so I’ll leave you with these:

Please contact your Senators and tell them that you do not support this regressive “reform”.