12 reasons public hackathons fail

Updated September 2019.
 

Disclaimer: No flowery language or elegant arguments; just a list of reasons why your company/organization shouldn't run public innovation hackathons.

Source/credibility: 8+ years of running epic hackathons with partners like Facebook, Amazon, Dove, Grindr, Deloitte, the Canadian/UK/US governments, etc.

TL;DR: Hackathons rock, but don't expect innovation or solutions to real problems.

 



What is a hackathon?


Many versions (e.g. design jams, business plan competitions) exist, but the core began as a computer programming competition. Just like how the word 'event' describes a wedding, a boxing match, and a funeral, the word 'hackathon' also has many applications. Hardware, internal, learn-to-code, infiltration, et al. For clarity, the 'hackathon' this article refers to is the popular 'public innovation hackathon'-type outlined below.

 

Typical public innovation hackathon characteristics

In-person event, overnight 24-48 hours, public call for participants from theme-related students/groups/mentors/mailing lists, coding only allowed at event, local pundit judges choose ranked winners based on 1-3-minute demos, token cash and prizes awarded, promises of funding/opportunities if teams create "something amazing". Hackathons are usually conceived from some variation of this logic:

"Tech solves problems in so many industries. Why not ours? Traditional approaches haven't solved our big challenges. Let's think outside the box and get fresh ideas from tech nerds not "held down" by industry standards and expectations. A hackathon does this by gathering 300 innovators for an inspiring weekend of pizza and code to solve our problems. We'll surely get at least 5-10 disruptive ideas and 3-5 brilliant teams building working prototypes."

This all sounds perfectly logical and reasonable, but reality tells a different story.

...sorry.

 

 

12 reasons not to run a public hackathon

(in no particular order)

1. Hackathons don't solve important or persistent problems.

Short timeframes force teams to pick the simplest, quickest, easiest thing they can build a pretty demo of and submit in time for judging -- or they won't win. Solving big, endemic problems takes time, effort, and understanding -- that won't fit into a 3-minute demo. If things were that easy to solve, they wouldn't be problems.

 

2. Output isn't likely to be "something amazing".

There's just not enough time/care/planning/incentive/motivation/sustainability. Particularly for teams that form "day of" - a hackathon is 1 hour of frantic ideation followed by 23 hours of typing and sleep deprivation... not a recipe for success. Zombies don't code or present well.

 

3. Winners aren't always winners; cheating is common.

For a big enough prize, some teams will build their demo "from scratch" 6 times leading up to the event -- good luck proving it. Others are actually fully-formed, product-already-developed startups looking for a good PR launch platform. It's easy to blow away judges when they're comparing 2-day hackjobs to a product you've worked on for 8 months. Journalists and investors love stories about geniuses who accidentally build amazing products and get 'discovered' at hackathons.

 

4. Hackathons are wasteful, inefficient, and demoralizing.

A big hackathon may have 300 participants working on complex problems all weekend. That's 14,400 hours of high-functioning human effort. Let's say 5% win prizes and praise. To the other 285 participants, the hackathon basically says, "Thanks, you weren't good enough to win. The 13,680 hours of hard work, effort, and good intentions you poured into the weekend amount to little more than 'a fun learning experience'. Go home."

Epic demotivation.

 

5. (Good) Hackathons get expensive; explaining this to people is an uphill battle.

A respectable Sat-Sun hackathon has at least 8 meals (if you're not fed at least every 6.5 hours, leave -- it's bad for your health/performance). Catering for 300 participants and 60 volunteers/organizers/mentors/sponsors at a very conservative $13 average per nutritious meal (excluding tax, staff, delivery) costs $37,440. Unless you're crafty and getting stuff donated, you'll also need to pay for venue, A/V, and furniture rental, staffing, major event insurance, prizes to convince 300 nerds to give up their weekends, ads, safe late-night volunteer transportation/housing, shirts and SWAG, banners and signs, extra cleaning, parking, salaries, and so on. A decent hackathon can easily run $150K+.

Side note: Eventually, some well-meaning cat who doesn't run events may want to properly educate you about what hackathons cost. They heard some company's amazing hack-fest-day-athon (where Kanye did a jazz flute solo at halftime and bought all the winning teams) only spent $15K -- they locked nerds in a room with pizza and caffeine and had a time travel app the next morning. A good strategy at that point is to excuse yourself and then try to find a window to climb out of.

 

6. Judges don't have magical judging skills.

The pundits, high-rankers, and minor celebrities that hackathon organizers recruit as judges generally lack the market/customer knowledge, experience, and intuition for any sort of calibrated demo evaluation. They may be department heads and CTOs, but unless they've actually evaluated hundreds of industry tech demos (like an investor), they have no clue what they're doing and it's 100% not their fault.

What happens when you go into a big parking lot and ask a non-driver to pick the car with the best handling? They pick the newest-looking sports car. Hackathon judging is the same.


Dirty secret: non-tech judges favor the prettiest/slickest presentation because they figure the 'polish' implies the team's technical competence.


Dirtier secret: tech judges do the same thing.

 

7. Judging is pretty ridiculous.

Ironically, hackathons as coding competitions judge projects on pretty much everything except code. Even with perfect documentation, proper code review is time-consuming and logistically unfeasible at standard hackathons. The 1 to 3-minute demo format makes it a contest of design and presentation skills, not technical prowess. Hackathons are won by designers and charming presenters; teams can win by making pretty mockups with little actual code because they know nobody looks under the hood.

After about 6 or 7 intense demos, the teams and demos start blurring together. Since judges don't take good notes (because organizers don't coach or set them up for it), they'll be confused and forgetful in deliberations, resulting in even-worse "judging".

 

8. Hackathon demo and judging processes are usually an afterthought.

Almost all hackathons feature the rapid-fire, one-after-another standard demo process to try to condense demo time while maintaining fairness. Here's some math. In a perfect world, each demo averages 7 minutes: setup (1 minute), demo (3 minutes), judge Q&A (2 minutes), takedown/transition/scoring (1 minute). Demos and judging for a small, 15-team hackathon (10-minute judge washroom break) will take 2+ hours. Without rock-solid processes, demos are sloppy and unfair.

300 participants, 5 max per team means 60 teams minimum (300 maximum). The unrealistic minimum of 60 teams doing standard demos will take at least SEVEN (7) HOURS. Imagine being on an exhausted team that worked 24-48 hours to shine on stage for 3 minutes -- and then has to sit through 7 hours of demos to find out who wins. Nuts, right?

Just to keep things manageable, HackerNest had to invent a new demo-and-judging system/process for big hackathons (100+ teams) that ran under 2.5 hours without reducing demo time. HackerNest Judging is thorough (teams can demo multiple times), transparent, and devastatingly effective. It's not complicated and runs very smoothly -- we're super proud of it -- but it took a lot to manifest.

Running a hackathon without having a fair, meaningful way for teams to showcase all their work is really terrible and, unfortunately, really common.

 

9. Participants aren't experts; mentors can only help so much.

Participants are clever and well-intentioned, but rarely subject matter experts. Mentors can help, but there's only so much they can impart in the 3 hours they spend with several groups of random people building things for fun.

There's a lot of, "Yes, CompanyX built the same thing you came up with 4 years ago, didn't you know?"

 

10. The overwhelming majority of teams don't (plan to) survive the weekend.

Teams don't expect to continue projects past judging. Most are formed ad hoc in order to participate -- whoever has calendar availability. Even if they win, they aren't likely to quit school/work and risk everything to spend the next 6 years toiling and building a stressful startup, managing a team, getting financing and fundraising, etc.

Teams break up, go home, and continue their comfortable lives.

 

11. IP is handled really, really awkwardly. And badly.

Fact.

Here's a fun scenario: CompanyX runs a hackathon. Teams participate, win, then break up after the weekend to go back to their regular lives. CompanyX likes the demo that TeamX did, so they decide to build it as a feature of their product. TeamX sees this and sues CompanyX for stealing their idea.

CompanyX laments, "Oh woe, if only we'd read HackerNest's article before running this crapathon, we'd have avoided this gerbillion-dollar lawsuit. Darn it, darn it all to heck."

 

12. Hackathons aren't great for ideation.

Traditionally, teams are pitted against each other and compete to win. Info and ideas get shared within teams but not between teams. So yes, you get different perspectives and that helps generate new ideas -- but they're limited to the people on your team. You're not really taking full advantage of the room by picking the brains of the 295 other clever people there. Also, reference item #9.

 

 


 

This should be enough to get you thinking. There are better ways to innovateIf you know someone who needs to run a truly amazing hackathon (or who needs to be talked out of one), we can help, go here for more information.