Yesterday, I had the opportunity to attend a meetup at GetYourGuide’s offices in Berlin, focusing on the practical application of Artificial Intelligence (AI) in the business environment. The theme was particularly centered around the recent developments in Large Language Models (LLMs) and Retrieval Augmented Generation (RAG), which piqued my interest due to my professional background and personal curiosities in similar technologies.

About the talks

The meetup featured three distinct talks, each shedding light on different facets of AI in production:

  1. GetYourGuide’s Support Slack Bot: This was the highlight of the evening for me, as it resonated with my work. The presentation delved into the challenges customer support agents face in swiftly accessing necessary information due to the overwhelming volume of documents. The solution? A sophisticated document pipeline that ingests content into a vector store, supported by a RAG architecture. What made this solution stand out were its innovative features:
  • Conversion of documents from PDF and HTML into Markdown for easier processing.
  • A user-friendly way to manage the ingestion pipeline through Slack emoji reactions.
  • Strategies to minimize chatbot fatigue with constrained auto-answers.
  • Efficient routing prompts to streamline response generation for frequently asked questions.

  1. Langdock’s Enterprise Search Solution: Langdock’s talk explored the construction of an enterprise-level search system using RAG. The narrative highlighted the common challenges faced and the independent development of LLM interaction tools for better control and scalability.

  2. Fun Talk on RAG and Political Documents: The final presentation took a lighter approach, applying RAG to political documents, offering an engaging and humorous close to the evening.

Observations and Learnings

The meetup was not only a platform for sharing innovative solutions but also a mirror reflecting the broader AI landscape:

  • A notable surge in non-funded or early-stage AI startups seeking founding engineers, reminiscent of the crypto and web3 bubble.
  • Despite the proliferation of AI tools, a significant gap in adoption rates persists, with GitHub Copilot being a notable exception.
  • A divide in perceptions between tech and non-tech participants, especially regarding the potential for AI to replace developers and the irreplaceability of jobs emphasizing human interaction.

Among the key learnings:

  • The effectiveness of overlapping chunks in indexing embeddings for better context retention by LLMs.
  • The versatility of Markdown as a semi-structured exchange format for LLMs, a practice I’ve leveraged in my side projects.
  • The complexities of managing permissions within semantic search systems, necessitating the indexing of metadata for appropriate filtering.
  • The reaffirmation that despite its advancements, RAG primarily remains a search problem.

Future Reflections

The discussions and presentations left me pondering two significant questions about the future of technology and labor:

  • To what extent will developers find their roles changing or even being replaced due to the advancements in AI?
  • As AI becomes more sophisticated, will jobs centered around human interactions and relationships withstand the tide of automation, or will they too see a transformation?

Conclusion

The AI in production meetup in Berlin was a profound gathering that not only showcased innovative applications of AI technologies but also sparked important conversations about the future of work in the tech industry. As we venture further into this AI-augmented era, the insights gained from such meetups become invaluable. They not only inform our current practices but also guide us in navigating the evolving landscape of technology and work.