Online events

WEBINAR "Inference Benchmarking of Prominent Open-Source LLMs"

Tuesday 27 February 2024

This event has finished.

Started 17:00 PM

Finished 18:00 PM


Organized by ODSC Lisbon Data Science


Venue: Online/Virtual

Address: Online event on your device
Portugal

See other Online events

Copy this link to share the event with anyone:


Share to social media:

About this event

**To access this webinar, please register here: [https://hubs.li/Q02h1Ndw0](https://hubs.li/Q02h1Ndw0)**

**Topic:** “Inference Benchmarking of Prominent Open-Source LLMs”

**Speaker:** Dr. Sanjay Basu, Senior Director – AI/ML at Oracle Cloud Engineering

*Dr. Sanjay Basu is an industry-recognized subject matter expert in Artificial Intelligence, Machine Learning, and Quantum Computing. He has double Master’s in computer science and systems design. His PhD was in Organizational Behaviour and Applied Neuroscience. Currently, he is pursuing his second PhD in AI with focus of research in Retentive Networks. Dr. Basu is also the author and editor of Ethics in AI collection, author of Web 3 books. View his latest blogs [here.](https://blogs.oracle.com/authors/sanjay-basu-phd?elqTrackId=78dd67b8199344cf8c93f65b4a6c7da4&elqaid=139889&elqat=2&source=:ow:lp:cpo::)*

**Abstract:**

In the upcoming webinar, we delve into the inference benchmarking of prominent open-source Large Language Models such as the 13B and 70B Llama-2. We have used a diverse range of compute shapes available inOracle Cloud Infrastructure (OCI), like Intel, AMD, ARM CPUs, and NVIDIA GPUs.

A core aspect of our discussion will center on the crucial metrics of Tokens per Second and the corresponding latency, which are pivotal in evaluating the performance of these LLMs. These metrics not only provide insight into the efficiency and speed of model inference but also serve as key indicators for optimization.

**Throughout the webinar, we will guide the audience through a comprehensive journey, covering the various stages of optimization for these LLMs.** *This includes an in-depth look at the unique challenges and solutions associated with each hardware platform. Our step-by-step process will highlight practical strategies and tweaks that can significantly enhance the performance of these models.*

**ODSC Links:**

• Get free access to more talks/trainings like this at Ai+ Training platform:

[https://hubs.li/H0Zycsf0](https://hubs.li/H0Zycsf0)

• ODSC blog: [https://opendatascience.com/](https://opendatascience.com/)

• Facebook: [https://www.facebook.com/OPENDATASCI](https://www.facebook.com/OPENDATASCI)

• Twitter: [https://twitter.com/_ODSC](https://twitter.com/_ODSC) & @odsc

• LinkedIn: [https://www.linkedin.com/company/open-data-science](https://www.linkedin.com/company/open-data-science)

• Slack Channel: [https://hubs.li/Q02hZrCD0](https://hubs.li/Q02hZrCD0)

• Code of conduct: [https://odsc.com/code-of-conduct/](https://odsc.com/code-of-conduct/)


This page last updated Monday 26 February 2024 at 14:00.

Problems? Report an error or inappropriate listing here.

Information displayed here is provided in good faith but we are not responsible for the content of any listing. Sometimes events can be cancelled or changed at short notice. Please check with the venue or organizer before you travel!

Oh no. Javascript is switched off in your browser.
Some bits of this website may not work unless you switch it on.