Organizing AI Engineer World's Fair 2024 #510
Comments
|
perhaps address some of the feedback received, and reflect on some of the challenges head on:
could be cathartic and calming haha |
|
Thanks for soliciting feedback openly! what worked
room for improvement
"bad ux"
crazy ideas
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
slug: aiewf-2024
We have just come off a very intense production period of the first AI Engineer World's Fair, the large, multi-track format of the AI Engineer series of conferences that my biz partner Ben and I are building. I am historically bad at writing down lessons and thoughts, so I am sitting down at my laptop on a friday evening at a McDonald's with my m2 macbook air on 16% battery and writing and writing until my battery runs out of juice (or I do).
venue
AI Eng Summit was 7-8 October 2023, and went fairly well. I was pretty set on us doing at least 2 conferences a year due to the pace of AI. As a frequent speaker and attendee of conferences I am inspired by large industry conferences like KubeCon (~10k attendees, cloud-native industry oriented) and NeurIPS (~15k attendees, research oriented), by big expos like Big Data London (~4-5k attendees? primarily expo hall with only small talks), and as a business by Infobip Shift (~4-5k attendees, bootstrapped and sold to InfoBip). All of these pale in comparison to -truly- big B2C conferences and trade shows, but they are generally landmark events for anyone in their respective developer sub-industries and it seemed obvious that we should do something like it.
The main problem was that Ben had never organized a conference larger than ~700 people before, and now we were about to go much larger. AIES 2023 was ~550 people; I just named a ballpark number of ~2k people as a goal.
I had always been inspired by the World's Fairs of a century ago, in my mind the canonical ideal of an expo of the best that the world has to offer, and in San Francisco if you have seen the Palace of the Fine Arts, or the Space Needle in Seattle, or the Eiffel Tower in Paris, or the China Pavilion in Shanghai, you've seen artefacts of former World Fairs permanently leaving their mark for modern cultures in a pale imitation of how ancient civilizations built megaliths and ziggurats for us to marvel at. So I also named the Palace of the Fine Arts as our ideal venue, and we launched the World's Fair -at- Summit itself, with "blind bird" tickets - fully refundable, no dates set yet beyond vaguely "Spring 2024".
The venue immediately turned out to be the sticking point. We did site tours of PoFA immediately after Summit. It had a beautiful theater that could fit ~1000 people, and a hall that could comfortably fit the expo and some stalls for food booths and food trucks. I thought PoFA could work because HuggingFace had hosted a ~2,000 person "Woodstock of AI" there a year ago and people seemed to love it. The Rotunda of the PoFA was a nice outdoorsey venue where we could hold a picturesque reception and perhaps open air concert (I was friends with a violinist who would love the opportunity to bring a string quartet). For the lake and park around the PoFA I even dreamed of having outdoor stands and booths and people flying drones around.
The reality was that PoFA is heavily regulated by the City of San Francisco. Outdoor stands would require extraordinary approvals we'd probably not get as an unknown conference. The Rotunda had historical significance and a very heavy extra cost to rent out. PoFA itself wasnt cheap but the real concern was noise leaking for a multitrack conference - we were looking to have 4-5 tracks, and our choices were smaller rooms of between 100-300 in capacity, that 1k person theater, and perhaps 1-2 areas where we could fit 300-500 people with lots of background noise. Because it was out in the Presidio we would also have trouble finding a hotel for guests and speakers to comfortably get to/from, and might have to charter buses. I was fine with it but we ultimately decided against it in the end.
By end January we had ended up going for our backup option, Fort Mason, which was set on the piers and had great views, and had the outdoorsey element I loved. We announced a Save the Date in end Jan-early Feb with a few visuals, and that helped kick off our early sales.
However, Fort Mason also didn't have enough stages for us to do multiple tracks, and we fretted about the cost going into March, because if we hit the low end of ticket sales projections it looked like we would be on the hook for very, very high fixed costs to the tune of 300-500k in losses, effectively bankrupting our business.
Venue needs to be played off against scheduling - I had my heart set on a Spring conference, and a 6 month pace would put us in April, and the concern was both cold weather and availability of the venue in short notice. I dont remember all the constant back and forth but we pushed and pushed until we ended up at a June date.
We sold "Blind Bird" tickets with no fixed venue the entire time - we had soft holds on both venues but could back out based on ticket sales and our other options. very late in the game (March?) Ben ended up finding the Marriot Marquis through an old contact of his, and eventually fell in love with their flexibility, ideal AV situation, site in the City, and the View Lounge that overlooked the whole city. I hated the idea of running the conference in essentially the air conditioned basement of a hotel, just like a billion other conferences before us, but Ben was enthusiastic about it and I gave in (it turned out not really to be an issue as far as I can tell, but I still maintain that outdoors-ness is more "fair" like and would have made for better social media).
With great venue requires great sponsor. I dont know if we can talk about the sponsor search and discussion in great detail, but our dream presenting sponsor was -always- Microsoft - they had supported Summit, were happy with it, and now we were about to ask them to trust us a lot more. They eventually agreed to do it, thanks to great champions from Britton on down.
launching
We launched the site with Msft Deputy CTO Sam Schillace on March 29. We had soft commitments to speak by then from all the major AI labs we wanted to feature (see below for more) so we were reasonably confident this would be a draw. but apart from venue and presenting sponsor (and a few others) we had basically nothing else locked in with 3 months to go - i did of course extend verbal offers to speakers i was courting for most of the prior 3 months.
By then I had also decided on the 9 tracks we wanted to feature, which led right into...
speakers
It seems a lot of people are interested in how we got speakers so I will recount as best I can... but the answer is going to be a bit boring. Just grind, make LatentSpacePod, have track record with AIE Summit, and ask people to come that you think will be interesting to your audience.
longer form:
We put out a Call For Proposals together with the launch, a traditional application process.
Amazingly, we got some incredible speakers INBOUND simply via this process, including 4 of our keynote speakers (Stephen Hood of Mozilla and Chris Lattner of Modular, Ishan Anand of Spreadsheets are all you need, and Danielle Perszyk of Adept - who eventually dropped out because of Adept's acquihire around the date of the conference), and many high profile/worthy speakers that we accepted straight into the conference:
These ~30-something folks were just ~6% of the ~500 that applied that passed our review/selections.
That meant at least about 30-50 more invited speakers that was entirely my job, apart from speakers that would be supplied from our sponsors.
I established a target list of dream speakers per track that I laid out, then set about inviting those people. I probably cant show you the list but basically imagine 8 candidate speakers x 9 tracks and me just spamming out every credible/worthy name I could think of. The biggest problem with doing 8 candidates was I didn't actually know what my "inventory" was - how many slots i had to offer - until very very late. something I probably should have agreed earlier with Ben, and caused a STUPID amount of undue stress and sorting trying to curate the right schedule while also not sending out invites that I would then have to retract if I overcommitted in any one particular track.
The most challenging were keynotes - we were variously trying to book Jeff Dean or Aidan Gomez or high up Microsoft, Anthropic, HuggingFace, and OpenAI person but their travel schedules were already fixed for those times. Some folks I reached out to initially accepted, and then became unresponsive, which caused extra chaos.
AI conferences need AI labs to bring the spice. We had OpenAI and Anthropic participation last year, so this year I needed the rest - Cohere, Mistral, etc. We got those thanks to their devrel and VC contacts. I made glaring omissions with Meta and Huggingface - which I basically have no excuse for but for just being stretched too thin.
AI conferences also need hot AI startups that poeple want to see. As you saw above we got decently lucky with our keynote CFPs already. Belle of the ball was Scott Wu/Cognition. Fortunately he had heard of me and we had chatted once - I just offered it and he accepted. not much to say there. I did similar opportunistic offers around Cursor, Cartesia, Dawn Analytics, Midjourney, Zeta Labs, Groq, and the 6 authors of the Year in LLMs post. Having an eye for what is "hot" and simply asking I think did most of the work here.
One thing I really tried to do was make the World's Fair representative of the world - I got contacts for African, Middle East, Indian and Chinese AI companies, and also invited contacts from Singapore and Japan. Basically, none of them bit, except for the Chinese groups like Qwen, who were interested but needed a 6 month lead time for visas. So basically we just got Europe and Latam in the end.
The other more challenging thing to handle was the AI Leadership track - something I had never organized before, a persona I wasn't particularly close to (and to a lesser extent the AI in F500 track as well). It turns out that I need not have worried - I attended an AI event by SourceGraph who gave me a lot of inspiration as to what appeals to AI Leaders and they helped me book a few people, then my natural network and the CFP (above) brought in the rest very organically. My proudest moment was inviting OpenAI, Cohere, LlamaIndex and Hex to do special sessions focused on gaps that I felt were missing, and they all accepted. Based on reviews, the AI Leadership track went particularly well.
One feature I created very late in the game was the concept of Track Hosts. I saw that Ben and I would be so stretched that we would never be able to ensure a good experience of the 5 simultaneous tracks, and having tracks without emcees felt bad. I again just rustled up my network to get the hosts together and most were happy to contribute, I think every single person I asked accepted (some actually preemptively volunteered, like Demetrios and Peter Cooper)
A final word on diversity. Last year we had 10% female speakers. This year I believe we had 50% female keynote speakers and track hosts, before the last minute changes we'll talk about next. Part of that was me putting my thumb on the scale slightly, but the inbound pool was genuinely more diverse and that of course organically helped the most. We topped it off by having my friend Sasha Sheng run our first ever Diversity committee, which yielded ~35 diverse attendees of the conference where we handed out scholarships, partially graciously supported by Writer who were the only diversity sponsors we managed to get.
the week prior
Marketing:
We aimed for ~2k attendees; on May 1st we had sold 25%, Jun 1st we had sold 41%, and only sold out on Jun 24th after Andrej's tweet. Here are some intentionally disaggregated line charts of the aggressive last-minute-fomo sales curve that make events like these extraordinarily hard to plan in advance:
For comparison observe how many years it took ICLR for their sales curve to reach an ideal 50% of tickets sold 60 days before the conference.
Content:
Many, many things are decided last minute. Every speaker is an external dependency, and every speaker themselves has dependencies we don't know about that could sway their commitment at any time, so its better for them the later we lock things down. A lot of ideal people - including OpenAI, Anthropic, Cohere, LlamaIndex, HuggingFace, Supabase, Exa, and others - only came inbound to speak last minute and we variously were either already full out of space or had to use them to backfill dropouts. And of course for our attendees and venue/printer partners they had the opposite need - everything should have been confirmed weeks earlier to be better.
This is how rough the schedule looked on Jun 17.
non-exhaustive list of things we did leading in...
what we did well
lessons learned
length and precision
looking ahead
Writeups from others
{% youtube https://youtu.be/JXsZrTEb6BI?si=IYuD8kbG_bTklwnC %}
{% youtube https://www.youtube.com/watch?v=uOBsacEoYKc %}
The text was updated successfully, but these errors were encountered: