SXSW Interactive 2015 Wrap-Up

Spring break has come and gone in Austin, which means that we’re recovering from another amazing SXSW Interactive festival. This year for me was a year of narrative story technologies and Community.  For the last several years I’ve been going to SXSW with my wife, Irma, and this year she had her own session.  That meant she spent a lot of time in the women in tech tracks, and we didn’t see each other as much as usual.  It’ll be interesting to read her write up, when she gets to it.

Friday – Al, Tim, BBQ, Old Friends, & 3D Printed Clothes

Friday started off with Life in the OASIS: Emulating the 1980’s in-Browser, a panel from Ready Player One author Ernest Cline (who has a new novel coming out, Armada, which I just pre-ordered), and Jason Scott, rogue librarian at the Internet Archive, talking about 80’s video games an in-browser emulators. Unfortunately due to our bus driver getting lost (transportation was a recurring pain-point at SXSW this year) I wasn’t able to make the session, but Jason, being a free-range archivist, put up the audio for all of us to enjoy.

Al GoreWhile we didn’t have time to get to the Life in the OASIS session, we had some time to burn till the session after, so Irma and I headed to Exhibit Hall 5, which is big and always has a lot of room to plop down and get your stuff sorted.  SXSW is the kind of conference where you can be just looking for a place to get your bearings and end up listening to Al Gore talk about climate change, the Pope, and his newfound optimism, which is exactly what happened.

After Al we moved up front for a presentation from perennial SXSW personality Tim Ferriss, who had a 30 minute How To Rock SXSW in 4 Hours talk, followed by QA. It’s always weird for me to see Tim at SXSW. I was on the first row of his first SXSW talk on the tiny Day Stage promoting the about to be released 4 Hour Work Week, way back in 2007.  To say our paths diverged would be an interesting understatement.  The main points of Tim’s talk were: Don’t be a jerk and treat everyone like they could make your career (because they probably can),.  He had some hangover cure suggestions (eat avocados before you go party), and reminded all the introverts to take the time to breathe.

Tim FerrissOne anecdote he told on the treating everyone well point was from one of his early CESes.  He spent most of his time in the bloggers lounge (a good place to meet people), and while everyone was trying to get the attention of Robert Scoble, he chatted up the lady checking people in.  Eventually he made a comment about Robert, and she said, “Oh, you should totally talk to him.  He’s my husband, let me give you a ring back in San Francisco and we’ll have lunch.”  So yea, you never know who people are.  I was standing in line at a session later in the conference and started talking to the lady next to me, who turned out to be the head of innovation at Intuit.

After Tim it was time for lunch, and we ended up at Ironworks BBQ. They have a $16.45 3 meat sampler plate (beef rib, sausage, and brisket), and well… a picture’s worth a thousand words…

Ironworks BBQ Plate

Our buddy Matt Sanders (formerly a Polycot, then an HPer, and now at Librato) was in town from San Francisco for the conference, and joined us to indulge in smoked meat.  We ended up eating at Ironworks at least 3 more times, which was kind of expensive, but fast and good.

After lunch Matt and I headed over to the new JW Marriott to a panel from Dutch fashion designer Pauline van Dongen titled Ready to Wear? Body Informed 3D Printed Fashion. This session was a perfect example of what makes SXSW such a unique conference: It’s a subject that I’m curious about, but one I’d never go to a conference specifically to see.  Pauline was wearing some of her tech-enabled fashions (a shirt with solar cells embedded in it), and talked about how fashion meets technology and how often in technology we design for the static (interlocking shapes), not the organic.  She profiled two of her projects, one a sleeve that morphs based on the wearer’s movement, and the other a neck ruff that uses electrically contracting wire to ‘breathe’ while worn. The challenges she faced (48 hour print cycles, unpredictability of material behavior) and insights discovered were really interesting, and it was one of the panels I kept thinking about most over the next few days.

After this panel I wandered through the job fair for a bit, which has expanded significantly in the last year.  It was interesting to see Target and Apple looking for candidates at SXSW.

Saturday – Storytelling Machines, Future Crime, New Parents in Tech

Talk Photo

First thing Saturday morning was my session with Jon Lebkowsky: Machines That Tell Stories.  We had a great turnout, and there are notes from the discussion at the link. Looking over the schedule, storytelling and storytelling systems were a very hot topic.  I was talking to Deus Ex Machina (an interactive theater project) producer Robert Matney later that it felt like the story zeitgeist erupted out of nowhere, and the flood of sessions made for a very interesting conference.  The discussion was really interesting, and it was gratifying to hear that there was a lot of cross-pollination between attendees.  I even heard that people were still connecting at the airport on their way home.

SocksChris Hurd, one of my friends and the guy behind DVinfo.net, gave me a tip one time from his years working big trade shows like NAB: The best way to keep a spring in your step at a long conference is to change your socks in the middle of the day.  So after leading our session, and stopping into the 3M booth, we went back to the car, dumped our stuff, and I changed my socks.  We had a long day ahead of us, and it was definitely worth it.

Next I went to a session titled Future Crimes From the Digital Underworld by Marc Goodman. It’s always interesting to see the people who’ve given a talk a lot of times versus people who are presenting the material for the first time.  Marc’s obviously really practiced at this talk, complete with jokes, audience-call-outs, and what have you.  It’s a fun talk, but the net-net is that everything in security is terrible, and it’s just going to get worse with trillions of IoT devices.  I didn’t need Marc to tell me that.  I have Taylor Swift.

After that was Irma’s meetup: New Parents in Tech. She had an interesting turnout, with only one other woman (there was a lot of competition for the women technologist this year, with a strong moms in tech panel opposite), but a lot of dads.  Two product guys from Fisher Price showed up, too, and I had an interesting discussion with them about Baby’s Musical Hands (Clara’s first app) and iPad cases (they don’t sell many anymore, possibly due to the kid market saturating with hand-me-downs).  After a good discussion, it was on to…

Saturday/Sunday – Community

Harmontown

Community!  And Harmontown!  And Dan Harmon!

Ok, I’ll admit it. Community is my biggest takeaway from SXSW. It’s my favorite TV show, the only thing I watch obsessively.  I’ve seen every episode, most of them a half dozen times. I follow the actors (even the lesser-known ones) on twitter. Fanboy = Me.

Yahoo! Screen picked up Community last year after NBC didn’t renew it.  The switch from broadcast to streaming distribution made it perfect SXSW fodder, especially after the Harmontown documentary premiered at SXSW Film last year.  Yahoo! pulled out the big guns, though, and beyond just having Dan show up to talk about the switch, they brought the whole cast, and premiered the 1st episode of the new season a few days early for the fans.  It was epic.

Community CastWe had great seats for Harmontown, and were next to the stage when they brought out the cast and showed the season premier. When they showed the episode everybody sat down on the floor, and in one of the most amazing things I’ve ever been in the middle of, we all sang along kumbaya style to the theme song.

The episode was great, and we had a wonderful time.

The next morning, after a panel I’ll talk about in a second, was a SXSW panel with the Community cast.  Everyone was there again, and there was a great discussion about the show.  For my money, it was even more interesting than Community’s previous Paleyfest discussions, probably due to the fact that there wasn’t a real moderator, just Dan Harmon asking the cast questions.  We had front row seats for that one, too.

Community Panel

Some notes for Community fans:

  • In the QA someone mentioned Conspiracy Theories and Interior Design as their favorite (it’s my favorite too, I was the weirdo in the audience who applauded at that), and Dan told a story about how they essentially threw out the last 1/3rd of the episode (the original ending involved the teachers creating the conspiracy) while they were shooting.  The final scene didn’t get scripted until they were in the study room blocking it off. They had NBC Standards and Practices on the phone, because there’s a lot of gunplay, and they were describing it, and finally the person from Standards and Practices said ‘Is there any way you can make it about gun safety?’  And if you’ve seen the episode, that’s how the ending happened.  Lots of Community episodes come together at the last minute.
  • The speech Ben Chang gives in Analysis of Cork-Based Networking (about the character being a real person, but just being portrayed as crazier and crazier) was lifted nearly verbatim from an email Ken Jeong wrote to Dan Harmon about the character. When Ken was performing it, he teared up (those are real tears) because he was so touched.
  • Alison Brie’s contract is up this season, we’ll see if she’s back if they make a movie or Season 7.
  • In discussing the longer episodes now that they aren’t constrained by NBC commercial breaks, Joel McHale noted that The Dick Van Dyke Show episodes were 29 minutes, which made me think that there’s an interesting comparison between The Dick Van Dyke Show as Community and I Love Lucy as The Big Bang Theory.

Ok, so that’s Community.  It was great.  Watch it on Yahoo! Screen.

Sunday – Transmedia Storytelling, Bot Authors, & Makers

Before the morning Community panel was a session titled Worlds Without Boundaries: Books, Games, Films, with James Frey (author and media maker) and Jon Hanke (architect behind Google’s Ingress game).  It was a fascinating discussion about media that crosses boundaries.  James mentioned he was heavily inspired as a kid by the book Masquerade, which included puzzles and a treasure hunt in the real world.  In his series, Endgame, there’s a puzzle pointing to keys that unlock a chest in the Caesars Palace casino in Las Vegas holding $.5, $1, and $1.5 million dollars in gold (per book, respectively).  They’re doing an app with Niantic Labs (Google), and they’re planning films.  It’s an interesting product development scheme: Have a stable of creatives come up with a world.  Sell some of the rights (film, TV), partner to do some products (games), and do others in house (books, novellas).

FloorAfter the Community panel I spent some time in the SXSW trade show.  General themes this year were lots of Japanese hardware startups (on Kickstarter, natch), lots of countries, and almost no hosting or cloud booths (save for Softlayer).  A lot more music, and a little ergonomic furniture.  Overall a less interactive-heavy trade show than years gone by.  I’m not entirely sure why that was, but there you go.  Wordpress didn’t come with their great t-shirts this year, so I guess I’ll have to actually go to the store to buy my clothes.  I did manage to pick up a new Olloclip, though, and even got to see it built in front of me.

In the afternoon I made it over to Automated Insights panel When Robots Write The News, What Will Humans Do?, moderated by James Kotecki, Automated Insights’ PR guy. This was an great discussion between Robbie Allen, the CEO of Automated Insights, and Lou Ferrara, a VP from Associated Press.  Automated Insights’ software produces AP stories in sports and stock reporting from raw data, and it was interesting to hear their discussion about what will be automated and where human value really lies.  Automated Insights thing is producing one billion pieces of content for one person each, which I think everyone can agree is where a large part of the content we consume is headed.

After the Automated Insights panel I headed over to SXSW Create, the free maker area of the conference.  While I was there I got to try out Lumo, a new interactive projector for kids that’s about to hit Indiegogo.  More on that later.

The Gaming Expo next door to Create was as crazy as ever, and really starting to outgrow the space they have for it.  The only larger space downtown is the Convention Center, though, which puts them in kind of a bind.  VR headsets were everywhere (almost always accompanied by lines of people waiting to use them), and it was good to see indie games like That Dragon, Cancer represented.

Monday – API Fails, Narrative Systems, Non-Linear Story Environments, Enchanted Objects, Home Projector Installations, & BBQ Science

The next day I tried to get into Wynn Netherland’s talk Secrets to Powerful APIs, but I was late, so Matt and I couldn’t fulfill our tweet-promise.

Instead, Matt and I went to the MedTech expo, and saw an interesting startup doing a small wireless body temp monitor for babies (slap a bandage over it, battery’s good for a month), some interesting sleep trackers (lot of quantified self folks at SXSW) and of course Withings with their smart watch whose batteries work for 8 months.

After that I headed to Technology, Story, and the Art of Performance by Elena Parker (who came to our Machines That Tell Stories session and had some of the best insights) and Michael Monello from Campfire, a division of Sapient Nitro that specializes in unique interactive experiences.

Elena and Michael’s presentation was one of the most meaty of the sessions I attended, full of helpful, hard-won insights into interactive projects.  Slides are available here, and audio is here..  They showcased three: Deja View, a project for Infinity where actors you see on screen talk to you through the phone (dynamic video branching and voice recognition), Hunted, an ARG-like that used some clever magic tricks to make users think they were being controlled, and a project for the From Dusk Till Dawn TV series, where players could call in and talk to the character Santanico Pandemonium and she would try to recruit them for her cult (branching narrative, voice recognition).

Some of my major takeaways from the presentation:

  • MaxMax: Elena talked about how while in games you program for MinMax (system constantly minimizes the players chances while maximizes the games chances, by attacking the player, moving enemies toward them, etc), in interactive story experiences you want to optimize for MaxMax, where you give the users as much of a chance of progressing as you can.  They’re likely only going to experience it once (replay value not being high, except for people who want to understand how the system works), so work as hard as you can to make sure they succeed.
  • Embrace Genre: When you’re giving people a new and unfamiliar experience, ground them in tropes and genre conventions they already understand. That way they have something to hold on to.
  • 3 Act Structure: Use the standard exposition, rising action, climax story structure.  Everybody understands it, it works, if you’re re-inventing all the other wheels, don’t re-invent that one.
  • Magic!: There’s an interesting cross-over with the magic community.  They worked with a magician to design some of their interactive tricks (powered, in the end, by people sitting in a call center). Fooling the brain is what they do, and is what delights our users.
  • Don’t Branch: Traditional Choose Your Own Adventure stories use a branching structure, which leads to some short experiences and some long ones.  That’s a negative for experiences you want users to fully enjoy, so instead of branches, create a looping structure where each act breaks into sections, but they all come back at the end.  Something like this:  -=-=-=-
  • Test: When you’re testing an interactive narrative, write out only the main 80% line first, and test it on 5 people.  This will validate your assumptions about how most people will view it, and won’t waste your time creating alternate paths if your base assumptions are broken.  Once you’ve passed 5, write out the rest and test on 50 people (I think this was how it went, they should post the slides soon) to validate your overall script.  Then run a production beta test on as many people as you can to get data for all the subtle things you wouldn’t expect.

Stories Asunder PanelAfter this panel I kept with the story theme and went to the Stories Asunder: Tales for the Internet of Things panel, with Lisa Woods from the Austin Interactive Installation meetup, and her team that produced Live at the Dead Horse Drum, an iBeacon powered non-linear location-based story experience on the east side of Austin.  Also joining them was Klasien van de Zandschulp from Lava Lab, who’s created some really fascinating geo-fence/iBeacon based non-linear story experiences in the Netherlands, including one under development where inhabitants of museum paintings create a social network the user can browse (think HogwartsPaintingBook).  They had some really interesting examples, and I hope to experience some of their work soon.

Enchanted ObjectsOn the way to my next panel I walked by the SXSW bookstore, and noticed that David Rose whose book Enchanted Objects I’d done a double-take on a few days before was going to be signing it just about then.  So I bought a copy and a few minutes later David showed up, and we had a great 10 minute long conversation about projection mapped interactive art objects.  David teaches at the MIT media lab, in addition to a lot of other stuff, and his book has moved straight to the top of my to-read stack.  It sounds exactly like a subject I’ve posted about here before, and something that feels like it’s moving from Bruce Sterling design fiction to real world product very quickly.

After talking to David I headed to the Storytelling Engines for Smart Environments panel, which had Meghan Athavale (aka Meg Rabbit) from Lumo Play on it. Meg’s been doing interactive projection installations in Museums for many years, and has had the question ‘Can I get this in my house?’ posed more than a few times. Recently component prices have been dropping, so she’s designed the Lumo Interactive Projector, a projector based toy for kids, and is about to run an Indiegogo for it.  Meg’s the kind of entrepreneur you can’t help but root for.  She came to SXSW by herself, set up a booth in Create, and is trying to drum up as much excitement as she can.  I really hope her Indiegogo is a big hit.

After Storytelling Engines I headed over to the GE BBQ Research Center with Matt and Irma for some free BBQ.  It was good, but Irma didn’t care for it.  Research accomplished!

Tuesday – AR/VR, Moonshots, New Assets, Space Cleaners, & Happy Bruce

Mixed Reality HabitatsTuesday morning I hit the Mixed Reality Habitats: The New Wired Frontier panel presented by IEEE. My biggest ‘wow’ takeaway, aside from the fact that nobody seems to know what Microsoft’s up to with Hololens or those Magic Leap guys (light fields?) with their headset, was from Todd Richmond, Director at USC’s Institute for Creative Technologies, who said his group felt that most people would be wearing headsets (Google Glass-like or Hololens AR like, or Oculus Rift VR like) 8 hours a day for work in 5-10 years.  When someone says something like that, I think it’s time to take notice.  Consider the headsets of today as the original iPhone.  Think about how far we’ve come in the 6 years since that was released.

After that I watched Astro Teller speak about Moonshots at Google [x]. His main point was that they always strive to fail quickly and get real-life feedback as fast as possible.  He talked about a bunch of wild projects they’re working on like delivery drones, internet by high-altitude self-driving balloons, kite-based wind power, the self-driving car, and others.  With each he emphasized how failure early lead to faster learning.

I hit a session entitled How to Rob a Bank: Vulnerabilities of New Money, with some fairly impressive speakers. Their main point seemed to be that your personal information is an emerging asset class that you should be concerned about.  That just like your dollars in a bank, your purchase history and address and Facebook posts have value, and we don’t really know how to protect that yet.

On our way back through the trade show Irma and I ran into Astroscale, a company from Singapore started by some Japanese ex-finance guys (follow me, here).  They’ve hired engineering resources to design a satellite that will de-orbit space debris.  Imagine that your $150 million dollar satellite is going to be impacted by a bit of out-of-control space junk.  You pay these guys $10 million, and then go find that space junk, attach their micro-satellite to it, and de-orbit it before it can crash into you.  And they’re running a promotional time capsule project with Pocari Sweat and National Geographic to collect well wishes from kids and send them via a SpaceX launch to the moon.  So yea, 30 years after Reagan’s Star Wars and Brilliant Pebbles, and here’s what we’ve got.  I’m surprised it isn’t on Kickstarter.

The end of SXSW is always Bruce Sterling’s talk, and this year was no different.  Bruce was kind of happy this year, and was almost channeling Temple Grandin in appearance, but happy Bruce isn’t always most interesting Bruce.  If you’d like to give his talk a listen, it’s up on SoundCloud.  Hopefully next year he’ll have some tales from Casa Jasmina to share.

So that was it, SXSW Interactive 2015.  5 days of old friends, new friends, stories, the future, BBQ, and space junk.  I can’t wait to see you next year!

Machines That Tell Stories: SXSW 2015

Last Saturday at SXSW Interactive Jon Lebkowsky and I curated a Core Conversation titled Machines That Tell Stories. I proposed the topic as a book project to Jon last year, and we put together this discussion as a stepping stone. Software storytellers are in the air. There were over a dozen sessions at SXSW this year on storytelling systems, and that kind of consensus usually heralds a new wave about to break. We’ve setup a twitter and tumblr for this project, if you want to follow along.

Machines That Tell Stories PlacardsOur argument: Software is moving beyond raw data and into narrative.  First it will help you weave the tales you want to spin, but soon it may be telling stories better than all but the best human storytellers.

The conversation was all over the place, and I don’t think anyone recorded it, but here are some notes and references that could be helpful…

IMG_8519

  • Lisa Cron’s Wired for Story: “A story is how what happens affects someone who’s trying to achieve what turns out to be a difficult goal, and how she changes as a result.”
  • Wired For Story Takeaway: Story is about mechanics, the trappings that you think of as important aren’t as critical as hitting the right beats that resonate with the human brain.
  • The Future of Storytelling Conference – Great speaker videos
  • Dwarf Fortress’s Legends mode, Procedural Poetry Analysis (Leave the creative imagination up to the user. Provide concrete, easy to procedurally generate elements, and let the brain fill in the rest.)
  • Weavrs as storytellers
  • The Nest Home Report monthly email as a machine-generated story
  • Collaborative human/machine storytelling at DARPA
  • Machine data into text reporting at Automated Insights (1 billion articles for 1 person each, instead of 1 article for 1 billion readers) More at CNN
  • Games by Angelina – Procedurally generated videogames, played through brute-force to see if they’re solvable. Potentially compare play throughs to known-pleasing physical interactions (progressively more complex button presses and movements)
  • Mechanical Turk as a part of a story machine, using human filtering to produce more compelling procedural content
  • Turing in The Imitation Game: The question isn’t whether machines will think like humans, it’s whether machines will think like machines.
  • tmbotg – Random TMBG tweeting bot, sometimes interacted with by humans due to serendipity
  • Talk PhotoWhy limit to text? Is software that generates a song based on your day’s quantified self data creating a story?
  • Shadows of Mordor’s Nemesis System as a storytelling engine – characters continue to exist when you aren’t looking, maintain the thread without you
  • Games as half-way points: PROCJAM’s The Inquisitor as procedural murder mystery
  • NaNoGenMo – Software generated novels
  • Eugene Goostman – Chatbot & Winner of the Loebner Prize.  13 year old Ukrainian boy personification: more constraints (space on twitter, language barrier with Eugene) result in increased credibility
  • Deus Ex Machina interactive theater project in Austin, sms polling to a web UI to allow for story decisions
  • Communal entertainment as a cultural touchstone: In a world where everyone gets personalized entertainment, does it become harder to relate to other people?  (No more watercooler conversations?)
  • Storium as a story generation human/software collaboration system

We had a great crowd for the conversation, and even managed to be “Hot” in the schedule.  Thanks to everyone who was able to make it!

Schedule-Hot

Data Day Texas 2015 Recap

Saturday was Data Day Texas (twitter), a single day conference covering a variety of big data topics up at the University of Texas’s conference center.  I went in my HP Helion big data guy role, and my wife Irma went as a python developer and PyLadies ATX organizer.  I’ve written up some notes on the conference for those interested and unable to attend.  As far as I know, there weren’t any recordings made, so this may be more useful than some other more archived conferences.

The conference was held at the University of Texas’s Conference Center.  It’s a nice facility, and probably appropriate for the number of people, but I think the place they hold Lone Star Ruby’s a little more friendly.  Conference organizers estimated the turnout at about 600 folks.  From what I saw, when presenters asked questions like ‘how many of you are x’, the audience breakdown was something like:

  • 70% app developers (not clear # of big data app vendors vs devs wanting to use big data)
  • 10% data scientists
  • 10% business types
  • 10% ops people

Big takeaways were that landscape immaturity is a big deal, and that’s forcing people to weigh trade-offs between the approaches they think are right, and the ones with the most traction (specific example was samza vs spark streaming at Scaling Data), because nobody wants to commit to building out all the features themselves, or getting stuck with the also-ran.  This is a problem for serious developers who want to architect or build systems with multi-year lifespans.  Kafka got mentioned a lot as a glue piece between parts of data pipelines, both at the front and at the back.  Everybody was talking about Avro and Parquet as best practice formats, and lots of calls not to just throw CSVs into HDFS.  There was a Python Data Science talk that ended on a somewhat gloomy note (the chance to build a core Python big data tool may have passed, and a lot of work will need to be done to stay competitive, slides at http://www.slideshare.net/wesm/pydata-the-next-generation).

The specific sessions I went to:

A New Year in Data Science: ML UnpausedPaco Nathan from Databricks

A talk that wandered through the ecosystem.  Paco’s big into containers right now.  Things he specifically called out as good:

The Thorn in the side of big Data: Too Few Artists by Christopher Re

A Few Useful Things to Know about Machine Learning by Pedro Domingos

He emphasized focusing on features, not algorithms as you develop your big data solutions.  Don’t get tied to a model, as our practices are all around proving or disproving models.  Build something that helps you build models.

Machine Learning: A Historical and Methodical Analysis (Historic, AI Magazine 1983)

He recommended the Partially Derivative Podcast, too.

Application Architectures with HadoopMark Grover

Related to the O’Reilly book: http://shop.oreilly.com/product/0636920033196.do

Mark talked about likely tradeoffs weighed in building a Google Analytics style clickstream processing pipeline.  Talked about Avro and Parquet, optimizing partition size (>1 gig data per day = daily partitions, <1 gig = monthly/weekly), Flume vs Kafka and Flume + Kafka, Kafka Channel as a buffer to ensure non-duplication, Spark Streaming as a micro-batch framework, and the tradeoffs of resiliency vs latency.  I think the clickstream analytics example is one of the ones in the book, so if this is interesting and you want more details, just buy an early access copy.

Beyond the Tweeting ToasterP Taylor Goetz

A general talk about sensors, Arduino, and Hadoop.  The demo was a tweeting IoT device, and Irma won it in the giveaway!

Real Time Data Processing Using Spark StreamingHari Shreedharan

Hari talked about Spark Streaming’s general use cases.  Likely flow was:

Ingest (Kafka/Flume) -> Processing (Spark Streaming) -> R/T Serving (Hbase/Impala)

He talked about how Spark follows the DAG to re-create results as its fault-tolerance model.  This was pretty cool, and an interesting way of thinking about the system.  Because you know all the steps taken to create the data, you can re-generate it at any time if you lose part of it by tracing it back and running those steps on that data subset again.  Spark uses Resilient Distributed Datasets to do this, and Spark Streaming essentially creates timestamped RDDs based on your batch interval (Default 2 seconds).

There’s good code reuse between spark streaming and regular spark, since you’re running on RDDs in the same code execution environment.  No need to throw your code away and start over if you want to do batch vs micro-batch.

Containers, Microservices and Machine LearningPaco Nathan

On the container and microservices front, Paco recommended watching Adrian Cockroft’s DockerCon EU keynote, State of the Art In Microservices.  He then walked through an example using textrank and pagerank as a way to create keyword phrases out of a connected text corpus (specifically apache mailing lists).

He mentioned databricks spark training resources, which look extensive: http://databricks.com/spark-training-resources

Building Data Pipelines with the Kite SDKJoey Echevarria

http://www.kitesdk.org/

Kite is an abstraction layer between the engine and your data that enforces best practices (always use compression, for instance).  It uses a db->table->row model that it calls namespace->dataset->entity.  He mentioned that they’d seen little performance difference between using raw HDFS vs Hive for ETL tasks, all things considered.  Use Avro for row based data (when you need context) and Parquet for column oriented data (when you need to sum/scan or only deal with a few columns).

Building a System for Event -Oriented Data by Eric SammerCTO of Scaling Data

A great talk on practical problems building large scale systems.  Scaling Data has built a product that essentially creates a kafka firehose for the enterprise datacenter, re-creating a lot of tooling I’ve seen at Facebook and other places, and making a straightforward-to-install enterprise product out of it.  They pipe stuff into solr for full text search (ala splunk), feed dashboards for alerts, archive everything for later forensics, etc.

He recommended this blog post by Jay Kreps at Linkedin on real-time data delivery mechanics:

http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying

Said their biggest nut to crack was the 2 phase delivery problem, guaranteeing that events would only land once.  They write to a tmp file in HDFS, close the hdfs file handle and ensure sync, then mark as read in kafka, then go process the tmp file.

Talked a lot about summingbird.  Said it was probably the right way to add stuff up, but that it was too big and gangly, so they’d written something themselves.  He recommended this paper by Oscar Boykin on Summingbird that covers a lot of the problems building this kind of system.

Also talked about Samza (best approach for the transform part of the pipeline, in their opinion, but low level and lacking community support), Storm (rigid, slow in their experience), and Spark (they hate it, but the community likes it, so they use it).

Wrapup

It was a harried (no lunch break, no afternoon break, if you were feeling burned out, you had to skip a session) conference, but that might be the nature of a one day brain-binge.  The organizers were happy to reserve a table for PyLadies in the Data Lounge, and they had a mini-meetup and got a little outreach done.

SXSW 2014: The One About Privacy

Kramer SXSW Cloud PlacardsTwo weekends ago SXSW Interactive graced our fair city, and as usual, I was there and even spoke a little.  Thankfully my house wasn’t robbed this time.

This year’s SXSW Interactive was heavy on privacy, internet security, and wresting our freedom back.  There weren’t keynotes from social players aiming to get you to join their thing, instead it was Edward Snowden, Julian Assange, and Neil DeGrasse Tyson telling you to learn and think for yourself.  It’s a refreshing change, and I’m eager to see what the tone of next year will be.

SXSW started really going on Friday this year, as the conference pushes up against it’s 5 day time window.  There wasn’t much in the morning on Friday.  Eric Schmidt, Jared Cohen, and Steven Levy had an interesting talk, mainly riffing on their book, The New Digital Age.  Eric and Jared are mainly concerned with technologies greater impacts, but there’s a certain large corporate mindset in what they say that clearly paints Google as a crusader for good.  You wouldn’t expect much else from the authors, but if you read it, keep that in the back of your mind.  There are other opinions.  One notable excerpt considering the Wikileaks presentation the next day were Eric Schmidt’s arguments against  transparency, which boil down to ‘Imagine what would happen if everything was transparent and open, nations wouldn’t be able to defend themselves from aggressors because they’d have to publish their attack plans before hand,’ which is just, well.  Ugh.

Show Your Work!Next up was Austin Kleon, who’s on tour supporting his new book, Show Your Work!  Austins keynote was the first one I really got something out of, my first big takeaway of the conference, which was that the concept of the lone creative visionary genius was patently false, and that we’re all products of the environment we’re in, and by showing your work in progress and getting involved and contributing, you can be a Scenius (hat tip to Brian Eno, there).  The people to avoid when you enter a creative community are the Vampires (people who feed off others energy to create their own work) and Human Spam (people who exist only to promote their thing, and are tone deaf to anything else).  Once SXSW pushes up video, Austin’s is a keynote worth checking out.

Julian Assange at SXSW 2014Saturday was Julian Assange, and you can watch the talk yourself here, but the long and short was that privacy is good, governments do bad things, people will act better if they know that what they’re doing will be made public later, and it’s impossible to do bad things on a large scale without creating a paper trail.  Julian is obviously a smart guy, but he isn’t a very dynamic speaker, and takes about three times as long to answer a question as he should.  He was in front of a green screen, and they use this constantly dripping wikileaks graphic that is incredibly distracting.  Pair that with his slow delivery, and it doesn’t make for a very exciting presentation.

Neil deGrasse Tyson at SXSW 2014There were exciting people at SXSW, though, and this years most exciting (to the point that he won Speaker of the Event), was Neil deGrasse Tyson.  Neil is a dynamic speaker, knows how to rally a crowd, and was there on the eve of the premier of his new series, Cosmos.  Takeaway from Neil is that science is cool, we keep learning new things, the universe is an amazing, mind-bogglingly-immense place and if you consider it in relation to our tiny planet and our tiny lives, it really puts everything in perspective the next time the kid is screaming and smearing blackberries on the table.

SXSW Gaming Expo

After Tyson’s talk we headed over to the SXSW Gaming Expo, which is a free event and a lot of fun for children of all ages.  They had a pretty big CCG pit, shown above, an area just for indie video games, and a big Gaming Tournament area where they were playing something I don’t keep up with anymore.

Escape the InternetSaturday night was the EFF-Austin SXSW party, In the future everything will work: Cyberpunk 2014.  It was a great time, with a cool Museum of Computer Culture exhibit of old machines and cutting-edge-way-back-when Hypercard decks.  Due to another commitment we weren’t able to stay late, but there was a panel discussion with Cory Doctorow, Bruce Sterling, Gareth Branwyn, William Barker and my buddy Jon Lebkowsky.  I’m bummed that I missed that, but you gotta do what you gotta do.

SXSW 2014 SessionSunday was my session, a Core Conversation titled A Cloud of One’s Own with me and Dave Sanford of the Austin Personal Cloud meetup group.  We had a really good turnout, and had a great, wide-ranging conversation on everything from Quantified Self analytics to Home Automation to crypto and authentication standards.  Dave prefaced the session with Life Simplified with Connected Devices, a piece of design fiction from the Connected Devices Laboratory at BYU.  It was written by Phil Windley’s son (Phil of Fuse connected car and Kynetx fame).

SXSW Bitcoin ATMAfter my talk we hit the trade show.  There was a bitcoin ATM this year, and NASA had a great booth with a 1/10th scale inflatable model of their new rocket that’s intended to take astronauts to Mars (this is Irma with a smaller model). There were some other interesting booths, including another good WordPress offering.  Irma and I grabbed some lunch, and while we were eating made our biggest missed connection of the whole conference, when only a sheet of glass separated us from Community’s own Troy Barnes, Donald Glover.  It was a very squee moment.  Donald was in town as Childish Gambino on his Deep Web Tour, and did a hackathon with WordPress.  We didn’t make it out to any of his events, which is a shame, but it was nice to have him here.

Edward Snowden at SXSWMonday started with Cory Doctorow talking with Barton Gellman about security and privacy and leaks.  It was arguably a more interesting talk than the actual Snowden interview that followed. The Snowden (video here), in a videocast with more technical challenges than Assange’s.  Assange was using Skype, though they lost audio.  Snowden was using Google Hangouts, but said he was bouncing through 7 proxies to get there.  The Snowden discussion was setup with two other speakers from the ACLU, so if he couldn’t make it, there would still be a talk, but this relegated him to a ‘voice on the phone’ role.  He’s very sharp, that Mr. Snowden, and knows how to make his point quickly.  It was an interesting contrast to Assange.  There weren’t any big bombshells from the Snowden interview, but it was an interesting moment in time.

After the Snowden talk I walked over to the Identity, Reputation, and Personal Clouds meetup session, as the organizers had been kind enough to come to ours.  We had a good discussion, and I ran into Chris Dancy, who I didn’t know before but seems to be the most connected man on the planet.  It was nice of him to come to the meetup, and we certainly had a rousing discussion.

Infinite Future PanelAfter the meetup was a great talk and interview with Adam Savage (I walked by someone I eventually recognized as Jamie Hyneman on the walk back to the Convention Center), I tried to get into the Infinite Future panel with Joi Ito, Bruce Sterling, Warren Ellis, and Daniel Suarez, but the room was tiny and way overbooked, and Irma couldn’t get in, so we headed back to the trade show instead, and then home.  I heard it was great, I hope there’s video or a recording somewhere.

Takei at SXSWTuesday was a quieter day, we showed up for George Takei’s interview, which was funny and interesting, though for someone who the US put into a detention camp, he has some interesting ideas on what Ed Snowden should do.  After that we bumped into our friend Carlos Ovalle, who’d been live tweeting the conference.  Carlos won my Personal Cloud SXSW Badge Contest the year before, and it was great to see the conference was good enough for him to come back.

After lunch it was Tim Ferriss talking about his new show The Tim Ferriss Experiment. He regaled us with stories of the brutal nature of shooting reality TV on a tight schedule, but the biggest takeaway I had was a quote from Jim Rohn on the law of averages, that: “You are the average of the five people you spend the most time with.” That really struck me, as I can see that pretty clearly, but it’s interesting to quantify.

Jon Lebkowsky Introduces Bruce Sterling at SXSW 2014The last session of SXSW is where Bruce Sterling gives his thoughts, and this year was no different.  My buddy Jon Lebkowsky introduced him (left).  Bruce’s talk was subdued this year, covering notable people who weren’t there (or were only there virtually), and who should be invited later.  It wasn’t a barn burner like it had been in the past, but I think Bruce’s thoughts were perhaps more with the things he is building and making, where he’s getting his hands dirty with real stuff.  It’s worth your time to track down some previous talks, though, because they’re great.

To close off, I will leave you with this, a photo of the traditional slice of Peanut Butter Mousse Pie that we had at Moonshine after SXSW wrapped up.  Moonshine and this pie is almost a tradition in itself at this point.

Moonshine Peanut Butter Mousse PieFollowing up on Austin’s idea of scenius and sharing your work, I’ve started a newsletter, Muniment.  I send it out every week or two, and preview new stuff that will end up on the blog, or put more context around interesting stuff I see.  You should sign up.

PyTexas 2013: 110 Miles to Aggie Land

Two weeks ago I had the pleasure of speaking at my first PyTexas conference.  I’d never been to PyTexas before, but I’ve been to it’s Ruby relative, Lone Star Ruby a bunch of times.  In a lot of ways it was similar (the local crowd, lots of enthusiasts, two tracks of talks), but in some ways, very different…

A&M Memorial Student CenterThe first and most notable thing to mention about PyTexas is that it’s held at the Memorial Student Center at Texas A&M University, which is in College Station.  That means the conference is two hours from Austin and Houston, and three hours from San Antonio and Dallas/Fort Worth.  This isn’t a complaint, it’s a nice facility, but it explains something about PyTexas: It’s not and will never be a large programming conference, simply due to being too far from the Texas programmer population.  That being said, it’s impressive how many people they’ve pulled in, and is a testament to the Texas Python community that so many people (about 100 folks the day I was there) made the trip.

The tradeoff for the drive is that the event (being hosted by the A&M School of Architecture) is really inexpensive ($25 early bird, $50 regular).  I would have thought that would have meant there would have been a big student turnout, but that didn’t seem to be the case.  School hadn’t started yet, so that may be one reason.  There were a lot of interested, engaged professionals there, and a lot of people doing serious day to day work with python.  I saw a couple of Rackers, and though there wasn’t anyone else I knew from HP Cloud, there was some OpenStack talk in the halls.

PyTexas RegistrationMy wife has been getting into python recently, and since I wasn’t planning on spending the night away from home (2 year old daughter + 7 months pregnant wife = at home at night), I talked her into coming with me for the day.  Registration was well organized, and there were good snacks.  The event had a few sponsors I wasn’t familiar with, including MapMyFitness, which tracks exercise metrics for folks, and StormPulse, which provides weather forecasts for businesses.  It’s always nice to see businesses showing how they’re using a language for real.  The Lone Star Ruby conference companies tend towards web startups and Rails.

HP Cloud StickersThe gender balance was about what you’d expect, maybe 10:1.  If it was a little bigger there might be a more organized outreach, but right now it’s just word of mouth.  I did hear about it on the PyLadies ATX list, and there may have been more women on the tutorial day.

I think there were some challenges on the organization side of the conference.  Speakers didn’t seem to get into the registration system, and two of the speakers didn’t show up.  That’d be easier to compensate for at a bigger conference, but when there are only two tracks it really shows.  Unfortunately one of the no-shows was Thomas Hatch of SaltStack, whose talk I was really looking forward to.  Maybe it’s online somewhere.

I’d proposed two talks, but only had time to prepare one, so I ended up spending 50 minutes talking the audience through building two simple Bottle applications.  One of the apps serves as an API service, the other as a web-exposed UI.  The code for both, built step by step with comments, is up on GitHub.  I’ll link to the video of the talk whenever they post it.

PyTexas Panorama

Walker Hale from the Baylor College of Medicine down in Houston spoke before me, talking about Bottle’s sister microframework Flask.  Flask and Bottle are really, really, really similar, so he stole a bit of my thunder, but I think the audience enjoyed the live coding I did (with paper diffs!), and I got some good feedback.  Unfortunately the Memorial Students Center is a no-hat building (out of respect for the Aggies who’ve given their lives in defense of the country), so the audience had to endure my out of control mop.

Docker Lightning TalkLunch was included in the cost of registration and provided by a nice local food truck.

There were a couple of lightning talks at the end of the day, including Barbara Shaurette of PyLadies Austin talking about her interesting new initiative to connect professional programmers with high school computer classrooms.  No set of lightning talks would be complete without the next big thing, Docker.io, so of course there were two (!) of those.  Docker’s going to take over the world, believe me.

PyTexas was a fun little conference, though driving down in the morning and back in the evening was really exhausting.  It’s small, and isn’t as slick as some larger conferences, but it has a nice raw charm.  The love the attendees and speakers have for python really shows through.  If it’s easy for you to get to, and you aren’t busy, I recommend it.  If they moved it to Austin or San Antonio, I’d go for the whole thing and I think the conference would be at least three times as big.  (Speaking of Texas python conferences, if you haven’t signed the Austin PyCon 2016/2017 petition, please do!)

OSCON 2013: The Source Must Flow

OSCON 2013It’s summer in Texas, which means one thing: It’s time to get away.  Last week I got away to OSCON, O’Reilly’s annual Open Source conference, in lovely, Portland, Oregon.  Herein is the account of that trip.

OSCON is a two and a half day conference preceded by two days of related tutorial sessions.  HP was a Diamond sponsor this year, so I finagled a free badge, and decided to go to the whole thing.  We didn’t have extra travel budget in my team, though, so I paid hotel and airfare out of my own pocket.  More on whether that was a worthwhile expense or not at the end of this post.

OSCON takes place at the lovely Oregon Convention Center across the river from downtown Portland, Oregon.  I lived in Portland for a year when I was in elementary school, and took a turn on the stage as Mr. Tumnus in Hinson Memorial Baptist Church’s production of The Lion, the Witch, and the Wardrobe.  I remember it being a lot larger than it apparently was.  That was a long time ago, and Portland’s a very different city now.

OSCON Networking RibbonsOSCON is a pan-technology conference.  As long as the project is Open Source, it’s welcome at OSCON.  Therefore you get a lot of variety, which is evidenced by the gigantic array of networking ribbons.  I didn’t stick one on, but I saw a few people with displays that would have made a Texas High School homecoming corsage maker jealous.

When I was picking tutorials I tried to focus on things I hadn’t gotten into before, but things I’d heard of, and wanted to know more about.  I ended up going for the R Predictive Analytics Workshop, Introduction to Go, Building a Distributed Sensor Network (with Arduino and XBee), and Erlang 101.  Some weren’t so great, some had unfortunate supply issues with parts.

Arduino AssemblyThe Distributed Sensor Network tutorial seemed really promising, but unfortunately we were missing the micro USB cables we needed to power our Arduinos.  Oh, and the Adafruit XBee Adapters we got were supposed to be pre-soldered, but weren’t.  Not an easy problem to solve when you have no soldering irons and only two and a half hours to do the whole tutorial.

The intent was to have an Arduino based sensor mote with temperature, humidity, IR-based movement and volume (sound pressure) sensors, which transmitted its data to a remote computer via the wireless XBee system.  Unfortunately we didn’t have the XBee adapters, and until half way through the class we couldn’t even power our Arduinos.  Fortunately one of the volunteers managed to run to Radio Shack and get us USB cables, but by then half the class was over.  We did manage to rig up a sensor to our Arduinos and get the data appearing via serial, and we have all the parts and the book with instructions to finish the project, but it was feeling like two and a half strikes in a row before I went to the Erlang talk…

Which was awesome!  Erlang is the weird friend you never knew you needed.  She does all the things that your other friends are terrible at, and after a long heart to heart at the local brewery, you totally get her.  Conference saved.  If multi-actor, highly scalable, multi-core programming is interesting to you, there are some great resources on its page, including Francesco Cesarini’s slides.

Erlang and Go seem to be two different implementations of similar ideas, trying to solve the massive concurrency problem in a structured, production-ready, robust way.  Go’s the hot new kid on the block, while Erlang has been in production for nearly 20 years.  Erlang seems to be a more interesting solution to me, though if you really like writing Java, C or C++, you might prefer Go.

You might have used Erlang if you’ve used CouchDB, Couchbase, Riak, Facebook Chat, Chef, RabbitMQ, voted in any of the UK Big Brother style SMS voting events, or ever sent data over a mobile phone network.  It grows across cores beautifully, and seems like it’ll be a really great solution when 64+ core processors hit the big time.  So, Erlang = Awesome, Conference Tutorials = Very Risky, Arduino Sensor Motes = Someday.

OSCON Space Party

Thursday’s opening party was space themed (I heard that last year it was Camp OSCON with merit badge activities and the like).  They had a jumpy balloon rig, space themed arcade games, interactive art, an indoor inflatable planetarium, a make your own space helmet craft table, and laser tag.  It was fun and loud, but the food options were limited for those on a diet, and as a non-social person, I soon wandered back to my hotel.

Every year OSCON has a nerd-oriented competitive activity.  Beat the game, win a prize.  This year the game was to collect 20 puzzle pieces (which you got from visiting booths, attending keynotes, having lunch, etc), and the prize was an OSCON 15th anniversary hoodie.  As a puzzle oriented and easily obsessed person I got my hoodie Wednesday morning, a few hours after the last piece had been made available.  I was somewhat disappointed to see that there were still hoodies available the last day, but I guess it’s good that those slackers were able to win, too.

Juju on HP Cloud at OSCONWednesday morning kicked off with keynotes, which were presented in an interesting, 10-20 minutes per speaker format.  One of the opening talks was by the president of Canonical, the company that produces Ubuntu and the cloud-oriented app orchestration system Juju.  He demoed Juju’s graphical cluster creation system running on top of HP Cloud, which was nice for us.  Juju looks like a neat system that compliments the existing solutions well, and it’s high on my list of things to look into.  There was also a great keynote about ‘My Robot Friend’ by Carin Meier, where she bravely did live hardware demos on stage, including a Clojure controlled quadcopter.

The most interesting keynote, though, was from Numenta.  Numenta’s keynote was presented by Jeff Hawkins, one of their founders and the guy who started Palm and Handspring.  Their technology simulates the neocortex, the part of your brain that remembers things and predicts patterns (specifically in their software, a 64,000 synapse slice of one of the layers).  They call it the Cortical Learning Algorithm, and they’ve open sourced it in the form of NuPIC (Numenta Platform for Intelligent Computing).  You feed data into this thing, and over time it builds up a map of the patterns in the data and can start to predict what will happen next.  The science is beyond me, but the demo and keynote was great, and you can (should) watch it on YouTube.  I went to their panel later, and they recommended Jeff’s book On Intelligence as a primer for those interested.  There are code samples (in Python!) with the NuPIC library up on their github account.

HP MoonshotThe keynote was impressive, and provided a nice start to the real meat of the conference.  While walking in I also happened to run into Pete Johnson, formerly of HP Cloud and now with ProfitBricks.  It was nice to see a friendly face.  HP also happened to have a booth in the trade show, doing demos of HP Cloud and showing off the oh-so-drool-worthy Moonshot Server.  (Drool worthy server shown at right.)

HP covered lunch for everybody on Wednesday, but I can’t remember what it was.  (I started doing a DietBet last week, so I only ate salads the entire conference.)  The conversations at lunch, though, were great.  On Wednesday I sat at a table with a Wisconsin lo-power FM radio and wholesale ISP guy, someone doing Hadoop at Disney (who’d previously worked at AWS), someone running a private cloud in Vancouver doing simulation-based pharmaceutical discovery, some guys from BlueHost (one of Code for America’s biggest sponsors) in Orem, and a guy who worked for an Apple accessory manufacturer in Portland.

The other panels I went to on Wednesday were one on the temporary cell phone network they setup up during Burning Man, a walkthrough of the parts and software needed to build your own cell phone with an Arduino (did you know that cell phone brains like the SIMCom SIM900 operate with an AT-command derived control setup, like your old 28.8 modem, including AT+HTTP commands to fetch web urls?), a talk on discreet math, and then one on getting kids to code (check out drtechniko.com, a robot language for kids to ‘program’ people, and Alice, a programmable machinima generator).  The last panel of the day was An Overview of Open Source in East Asia, with some really interesting insights into the Open Source community in China, Korea and Japan (and they gave us all free fans!).

OSCON OpenStack 3rd Birthday Bash

OpenStack Birthday Bash3 years ago at OSCON the OpenStack project made its debut, so that means it was time for a 3rd birthday bash.  Fellow HP Cloud-er Rajeev Pandey and I walked over, enjoyed some gazpacho shots, picked up a t-shirt or two, and marveled at their giant paella (seriously, they were like 3 feet wide).  We ran into a few other HP Cloud folks there, including Monty Taylor.  There was a cute birthday cake and lots of cupcakes, but after nibbling and conversing and drinking lots of water (it was surprisingly warm in Portland), soon it was time to go.  Happy Birthday, OpenStack, in software years you’ve almost hit puberty.

The Thursday morning crowd was a bit more subdued, with a fair number of attendees probably partying a little too hearty the night before.  Keynotes were good, with a great talk about Technology diversity by Laura Weidman Powers, co-founder of CODE2040. Licenses were a hot topic as well, including a talk about licenses effecting communities from HP’s own Eileen Evans.  It’s hard to top brain simulation and flying robots, though.

Docker StickerThursday I attended Tim O’Reilly’s talk on Creating More Value Than You Capture (and as an aside, I felt both sorry for Tim in only getting 30-40 attendees, but also better about the 15 my talk pulled in at SXSW), and a great intro to Docker from dotCloud.  If you haven’t looked at Docker, check it out.  The way they bundle up app binaries on top of base machines is awesome.  Then came lunch, with another great group of folks including someone managing DevOps for Disney.com (the entire thing on 60 VMs!).

After lunch was a really great talk on Kicking Impostor Syndrome in the Head by Denise Paolucci.  If you ever feel insecure about your skills, dig up a video of her giving that talk, it was really great.  After that was Designing the Internet of Things with the 3 Laws of Robotics, and then From Maker to China, where Brady Forrest described the challenges and pitfalls of taking a concept from prototype to small-scale manufacturing in China.  One book he recommended for those interested in the product design and manufacturing process was From Concept to Consumer, which now rests on my Amazon wishlist.  After that it was Hardware Hacking with Your Kids, with some funny slides and interesting anecdotes from Dave Neary, and then we were done for the day.  That night I worked on my SXSW panel proposal, and went to bed early.

Trade Show Caterpillar Head

OSCON SwagThe trade show went on Wednesday and Thursday, and had a good mix of big companies, lots of non-profits, and some interestingly unexpected exhibitors (League of Legends maker Riot Games).  There were some great shirts, including this Cloudera one: Data is the New Bacon, and its sister, Data is the New Tofu, one from the Kenyan data mapping non-profit Ushahidi, and plenty of other knicknacks and stickers for the kids back home.  PyLadies was there, Wikimedia was there, Craigslist was there, FSF, EFF, and the Linux Foundation were there.  Everyone was hiring.  The Tizen folks are giving away $4,040,000 (that’s four, count em four… million…) dollars in app development prizes.  There were more hosting and big data software companies than I have fingers and toes.  It’s a good time to be in technology.

Friday was only a half day, so after a keynote exhorting us to join the ACM, one noting that everything important has already been invented, and some group singing, we settled down to business.  First up was Cryptography Pitfalls with John Downey of Braintree.  That was a great talk, and though I knew a lot of the gotchas he mentioned, it was still nice to hear them laid out by a professional.  In short: Use a slow one-way hasher for passwords, don’t build your own crypto implementations, and always check SSL cert validity in your application code.  The slides are up, you should take a look at them.

OSCON ChalkboardAfter a break we headed into Open Source Social Coding for Good, with Benetech.  I’d run into the folks from Benetech in the trade show the day before, and was really excited to learn that they were doing hackathons already with HP’s Office of Global Social Innovation through their SocialCoding4Good project.  I’m really hoping to connect both of them to HP Cloud and do a hackathon in Austin.  The panel was great, and it was good to hear about nonprofits getting traction from corporate hackathons and volunteers.  We need to do more of that.  After that it was Polyglot Application Persistence, and then the conference was over.

So, back to my original question, was it worth it?  Would I go again?

If you’re in Portland, or the Portland area, I think it’s a no-brainer.  It’s a great conference, the attendees are sharp, it covers a ton of stuff, the keynotes are good, and I’m sure there’s something interesting every year.  The trade show’s great.  If you can’t snag a speaking slot or a super-discounted badge, you could get a lot of the value by getting an expo badge and watching the keynotes online.  If you’re paying for it yourself, and traveling to do it, it becomes a much murkier question.  So many conferences are putting everything online these days, what you’re really paying for are the networking opportunities and the experience: That conference euphoria of anything is possible.  That has a lot of value, but if you’re on a budget, maybe local conferences, hackathons, or meetups are good enough.  I hope I’ll be back at OSCON next year, but if I’m not, you’ll all just have to have fun without me.

HP Tech Con `13: The Magic Kingdom

After I joined HP two and a half years ago I started to hear tales of a magical event.  Tech Con, a technical conference for the top technologists in the company, showing off the best and brightest innovations of the year.  Moonshot servers, 3D displays, that kind of thing.  One guy on our team had gotten in on an honorable mention a few years before.  Nobody ever went to present.  Not from our small corner of HP, anyway.  But we could always dream.

To get in to Tech Con you have to write up an innovation you created, and then compete against nearly two thousand other proposals for one of less than 150 poster slots.  Out of those poster slots, fewer than 50 get speaking slots.  To put this in perspective, HP has 320,000 employees, more than 70,000 technologists, less than 1,000 get to go to the conference and fewer than 50 get to stand up on stage and talk.  HP Labs is always well represented, innovation is their lifeblood.  HP Cloud… had never sent anyone.

PodiumLast year I started working on something pretty cool, so a logical step while applying for a patent was to submit the innovation to the conference.  Fast forward a few months to late February.  While sitting on the couch at the end of a long day, I was checking my email on my phone and a message popped up.  It was an invitation to Tech Con.  Even more mind boggling, after I re-read the email for the third time, I realized I’d been invited to speak.  Queue the montage of presentation creation, practice, tweaking, throwing the entire presentation away, starting from scratch, practicing, tweaking, etc.

You can read about what other HP People higher in the company than me have to say about Tech Con: HP’s CTO Martin Fink highlighted it on the HP Next blog, and HP Fellow Charlie Bess has posted about it.  For those who haven’t had a chance to go to an event like Tech Con, a big, technical company event, I’d like to give a brief rundown of what it’s like, since it can be a once in a career experience.

Room KeyTech Con this year was in Anaheim.  The location is a closely guarded secret before the conference, because it’s chock-full of company trade secrets and unreleased products.  We were at a hotel a few minutes from Disneyland, and HP had booked nearly the entire thing for the conference.  There were a few vacationers, but pretty much everyone you saw was wearing the HP Tech Con lanyard.  Even our room keys had the conference logo on them.

LunchTech Con is a conference where you work hard and play hard, in as compressed an amount of time as possible.  We all flew in Sunday afternoon, and then had a reception and dinner that evening.  Being a teleworker I don’t get to see HP at it’s most mind-bogglingly huge, but this conference went all out.  HP logos on the walls, HP banners, even HP napkins.  When you work in a little 350 person department in the cloud it’s easy to forget that the company makes paper, ink and printers of all kinds.  (Not to mention servers, PCs, storage, networking, software, services…)

The conference is a mix of formal presentations and academic style poster sessions.  During the poster sessions my co-authors from HP Cloud and I got to stand in front of our poster and talk to technologists and executives from all over HP, discussing how we could work together, explaining our product and getting some really great feedback.  HP’s COO (and our boss’s boss’s boss’s boss) even dropped by our poster and talked with us for about 20 minutes.  It was amazing to get that level of recognition, and I think I managed to not completely embarrass myself.  Meg Whitman was on the poster hall floor, but unfortunately didn’t get to where we were.  She did talk to all of us, though, which was great for those of us who aren’t from the bay area and don’t see her regularly.

Tech Con Dinner

There were two blocks of scheduled activities, the first an outing to the Queen Mary that I missed to spend more time with some fellow HP Cloud folks (though everyone I talked to said it was really fun), and second an activities slot with things like whale watching, art walks, and the Warner Brothers studio tour, which I went on.

Nerd Herder

The highlight of that tour might have been seeing a real Nerd Herder from the TV show Chuck, but we also got to see the main set for The Mentalist, which was really cool.

Speaking at Tech ConI gave my talk Wednesday morning to a nice, big crowd, had some great QA, and then talked to a bunch more people at our final poster session.  It was great to hear from people who’d had 30 year careers at HP working on amazing products that they totally got our thing and thought we were on the right track.  That’s the kind of affirmation that can only come from a smart, diverse group of people like those at Tech Con.

Tech Con Room

Tech Con is a highly confidential conference, and they take IP violations really seriously.  I wanted to bring my poster home, since it was really pretty, and would go really well with my Whole Foods Market Milk poster.  I had some people ask about the possibility on my behalf, but apparently after the last session they were going to lock the door, ship them all back to Palo Alto, show them for a few days, then cut them up, boil them in acid, burn them, and bury them in the desert.  Oh well.

The conference wound down Wednesday afternoon, but I managed to walk over to Disneyland with one of my coworkers from HP Cloud to get souvenirs for the family, having a nice chat on the way.  The highlight of the conference is really those times, getting to talk to people you work with, or people from other parts of HP that you never have contact with.  I had conversations with people from Australia, the UK, Germany, India, China, Italy, Israel and Brazil.  It was really, really cool.

PyCon 2013: Three Days in the Valley

Last weekend I was in Santa Clara for PyCon.  Since then the story of the conference has been writ large in media outlets near and far, but you may not have heard anything about what the conference was really like.  So here’s my view, as someone who had never been to PyCon before (with some thoughts on the controversy interspersed)…

HP Cloud was a sponsor and exhibitor this year at PyCon.  I’m working on a new cloud service written in Python and will need developers at some point, so I traded manning the HP booth for a few hours for the trip.  I’ve been to Lone Star Ruby a few times and two RailsConfs, but I’d never been to a Python event.  Given Python’s reputation as a very friendly, open community, I wanted to get a feel for it it in person.

I’ve never been to the valley proper.  I’ve been to San Francisco a couple of times, but never down to Palo Alto, Mountain View, Menlo Park, Sunnyvale, Santa Clara, Cupertino, San Jose, and surrounds.  In tech, Silicon Valley inhabits a mythical place as the fount from which innovations flow.  Books have been written about how special the place is.  Barrels of digital ink have been spilled over the high cost of living, the startup life, and the bright lights up the 101 in the City.

The AvatarI flew in late Thursday night after a crazy week attending and presenting at SXSW, and then getting robbed.  On the approach vector into San Jose International the whole valley spreads out beneath you, tight, flat grid of civilization.  It’s very Tron.  After taking a taxi from the airport, the only thing that really struck me was that every building I saw over one story had the logo of a tech company I knew on it.  I didn’t book travel in time to get into the convention center hotel, so I was in The Avatar, the overflow hotel.  The Avatar is an 8 bit/robot themed hotel, but really it’s a refurbished 1950’s motor style Holiday Inn with some modern furniture.  At check-in there was a lady in front of me with green hair and big black boots, and in my post-travel haze, surrounded by tin robots and chrome, I remember thinking that this must be where all the cypherpunks had gone.

In the morning light, Santa Clara looked a bit more like every tech town USA, though there was still that ineffable California sheen.  I took the overcrowded bus over to the convention center, picked up my badge, and had a very nice breakfast.  It was a standard eggs and bacon affair, but they were pretty liberal with the bacon.  I think I saw a guy whose entire plate was bacon.

Keynote RoomI picked up my conference bag from a guy wearing a Wreck it Ralph tech team shirt.  Apparently Disney Animation was a sponsor this year.  Next up was the keynote from Eben Upton, where they announced that everyone was getting a Raspberry Pi.  There was a lot of cheering.  He also said that originally they were hoping to make a device that booted straight into python, so if you wanted to do anything you’d need to learn to code, ala the C64 and BBC Microcomputer.  The Pi in Raspberry Pi was originally for Python.  They’re still working on that idea.  The organizers also mentioned in the announcements about the Young Coder program they ran, with obligatory adorable pictures of kids peeking out from behind monitors.

The sessions were interesting, and since it seemed they were being recorded, I didn’t feel as much pressure to sit in every one that seemed cool.  The Messaging at Scale at Instagram talk was really interesting, as was the Making DISQUS Realtime talk.  It’s pretty incredible the traffic the DISQUS folks are pushing out of a half dozen physical boxes.  Whenever you’re on a page with DISQUS comments and you see one slide into the live comments box, you’re talking to one of those half dozen machines.  Crazy.  They had some interesting traffic graphs from the week the new pope was announced.

The Pope

After a few panels I decided to hit the trade show, which really surprised me.  It’s a good time to be a Python programmer.  The trade show at PyCon, a conference of only 2,500 attendees, was one of the best I’ve seen.  I’d never seen a trade show with Facebook, Oracle, Google, redhat, eBay, Microsoft, Amazon, Twitter, Apple, Netflix, Firefox, Hulu and of course, HP Cloud, all in one place.  We sponsored a happy hour the first day, and Heroku covered the second day with free sake.  There was even raspberry pi(e).

Lunch was really well organized, with 7-8 two sided serving tables and acres of big round tables.  The food was ok, nothing to write home about, but better than some conferences I’ve been to.  Breakfast was really their forte, the second and third days we had really satisfying baja breakfast burritos.

Lunch Lines

One of the trade show vendors, Thumbtack, a company that offers custom local service quotes (and is an awful lot like a site we worked on at Polycot, 45fix), had a programming challenge they were handing out.  I’m afraid to say that I burned more than a couple hours over the weekend working on it, and in the end I ended up with a fairly brute force approach that I wasn’t entirely satisfied with, but seemed to be the only straightforward way to solve it.  The programming challenge pages are here, if you’d like to take a crack.  The solution to the second page challenge ended up taking around 25 seconds on my Macbook Pro:

Thumbstack 1 Thumbstack 2

Donglegate

So let’s get into some controversy, shall we?  The Python community is known as an open, welcoming community.  Like any programming community there are plenty of hard core nerds who like to prove how smart they are, but Python was designed as a language that would be very consistent and easy to learn.  There was an entire track on how to teach python, how to run meetups and events, and how to get more women coders into the community.  PyCon has a code of conduct as well, something that attempts to directly address previous inappropriate activity in the programming community.  The Python leadership and organizers want to be really welcoming, they want a good gender balance, they were even talking about how the conference attendance was 20% female.  I think this number is probably skewed because it probably includes a lot of marketing folks who were only manning booths in the trade show, but they’re definitely trying.

There were at least 5 female programmer groups in the trade show: PyLadies, Women who Code, LadyCoders, CodeChix and The Ada Initiative.  There was a charity auction for PyLadies, and the Ada Initiative even had a feminist hacker lounge in the trade show:

Feminist Hacker Lounge

It was by far the most actively gender progressive conference I’ve ever been to, which makes the whole hullabaloo about dongles and forking so weird.  There was a lot of justified outrage after the Golden Gate Ruby CouchDB talk.  The Ruby community isn’t known for being as newbie friendly, and is generally a bit more rock star testosterone driven.  PyCon tried to do a better job, and despite all their good efforts, the takeaway from most of the people who read about the event will be, “Won’t those nerds ever learn to treat women with respect.”  That’s a shame, because they really tried.  If you’re interested in diving into this rabbit hole, the Geek Feminism wiki has a good page about it.

I keep thinking that the gender equality thing that PyCon tries to promote is a lot like the friendliness of the community.  It exists because we say it does, and the fact that there’s a conversation around it makes it real.  If you’re sitting next to someone at a conference that talks a lot about friendliness, you’re more likely to be friendly and open yourself up and risk rejection.  I had a lot of great, interesting conversations at PyCon over breakfast and lunch, including one with a young lady from Portland who had been to PyLadies and other female programmer meetups.  She said what she really wanted wasn’t get togethers to talk about how being female in tech is weird, she wanted meetups where they sat down and actually wrote code.  She said that if programming is a meritocracy, you should be able to prove yourself and grow by doing, which makes sense to me.  Less dongle jokes, more ladies, more kids, more code.  It’s a big tent.

Right after registration I was standing next to a group of people who had clustered together, and someone actually invited me over to join the conversation.  I’ve never had that happen at a tech conference, ever.  It turned out that none of the people in the group had ever been to PyCon before.  It wasn’t a passed down openness based on previous experience, it was because we all knew PyCon was open, because they make a point of saying it.  It’s right there on the conference web page: “Change the future – education, outreach, politeness, respect, tenacity and vision.”

I don’t have a good answer on how this whole thing should have played out.  It’s a mess.  It shouldn’t have been a mess.  I hope the folks who organized PyCon aren’t taking it too personally.  I don’t see that they could have done anything better than they did.

Booth Monkey Like Me

I went to PyCon, in part, to man the HP Cloud booth.  The last time I manned a booth was at SXSW, where while covering for the Creative Commons folks during their session, Bruce Sterling walked up to me and asked why he should give his books away for free.  I didn’t have a good answer.

Booth Monkey

This time was a little easier, the thing we’re battling the most with developer at HP Cloud is just awareness.  Most people don’t know that HP has a public cloud offering, so I was happy to explain what we did and get some insights from real customers.  Of course, the Spotify booth was opposite ours, and getting those insights can be a challenge when you’re competing with this:

Wrapping Up & Going Home

I never got to really see much of Silicon Valley.  I didn’t get to hit the Apple Company Store or visit the garage or the HP offices in Palo Alto.  Hopefully I’ll be able to go back soon.

There were some other really good talks at PyCon.  I know I need to start using iterators and generators more.  I may even take a poke around Python 3.3.  On Sunday they had a job fair and poster sessions, which was really interesting to me, since I’ll be presenting a poster in a month and a half at an HP conference.

PyCon Job Fair & Poster SessionsRecruiting was the activity of the conference.  It seemed like everyone was looking for Python developers, and like Ruby was back in 2007, there just aren’t enough to go around.

Bag-O-Swag

When I flew out to Santa Clara I only had my laptop bag.  Walking around the trade show I realized that I didn’t really need to bring extra t-shirts, nearly everyone was giving them away.  I ended up carefully packing an entire bag of swag, including my hard-fought goodies from Thumbtack.  Thankfully the San Jose airport’s bathrooms have child seats.  HP had some nice swag this year, a pen-shaped screwdriver set.  Someone even came up and gave me a compliment about it.

I decided to get some Python neckerchief wearing beanie snakes for the girls back home, which gave me a chance to take this picture.  I have had it with these pythonic snakes on this pythonic plane!

Pythonic Snakes on a Pythonic Plane

Austin’s a big tech town, so it wasn’t a surprise that I ended up sitting next to a fellow PyCon attendee.  In this case it was Chris Kucharski, the guy who runs the web team at Dimensional Fund Advisors.  We had a great chat about Python, Austin, teams and technology.  It was cool to find out that he hosts the Austin Learn Python Meetup at Dimensional’s offices.  The more supporters in the community and the more new developers, the better.  Maybe in a few years PyCon will be as diverse as we all want it to be.

Experiencing TEDxAustin 2013: FearLess

Yesterday I was lucky enough to attend TEDxAustin, the fourth independent TED event here in town.  I’d applied to attend before, but this year I finally made the cut.  If you’re interested in what it’s like to go to a TEDx event, hopefully this post will shed some light.  If you’d like to watch the presentations check out TEDxAustin.com or their archived livestream.

Continue reading “Experiencing TEDxAustin 2013: FearLess”

OpenStack Austin February Meetup Videos

Last night was the February OpenStack Austin meetup.  I took my handy little Canon S95 with me to record the proceedings for those of us who couldn’t make it.  Here are the two videos from last night, and a special bonus video from December’s meetup.

Unfortunately the S95 doesn’t handle auto-exposure well with the super-bright projector image, so the camera keeps under and over exposing these videos. Hopefully it won’t be too distracting, maybe next time I’ll bring a camera with more manual control.

First, Matt Ray from OpsCode talked about the history of OpenStack and Chef and the knife tools for managing OpenStack. YouTube Link

Zaid Sawalha from Rackspace talks about how OpenStack Keystone became an incubated OpenStack project, and lessons learned from their experience. YouTube Link

Zaid Sawalha from Rackspace talks about OpenStack Keystone’s implementation and development, plus a little QA. YouTube Link

Bonus! Blake Yeager from HP Cloud talks about Deployment strategies at the December 8th meetup. YouTube Link