NVIDIA TensorRT
NVIDIA TensorRT
Apr 5, 2016
nvidia.com
What is Quantization? | IBM
What is Quantization? | IBM
Jul 31, 2024
ibm.com
Quantization Aware Training with TensorFlow Model Optimization Toolkit - Performance with Accuracy
Quantization Aware Training with TensorFlow Model Optimization Toolkit - Performance with Accuracy
Apr 8, 2020
tensorflow.org
Quantization: What Everyone Gets Wrong (Accuracy Myths)
0:44
Quantization: What Everyone Gets Wrong (Accuracy Myths)
65 views3 weeks ago
YouTubeCode & Capital
Google magic bullet - TurboQuant #ai #gpu #google #chips #cuda #quantization
0:41
Google magic bullet - TurboQuant #ai #gpu #google #chips #cuda #quantization
1.3K views1 month ago
YouTubeNeural AI Flair
Optimizing LLMs with TensorRT Post-Training Quantization
7:01
Optimizing LLMs with TensorRT Post-Training Quantization
3 views2 months ago
YouTubeMosaic Flow
FPS GPU Optimization Tokens #tokenization #llama #nvidia #ai #rtx #gpu #gaming #gpublock #nvidiagtx
0:33
FPS GPU Optimization Tokens #tokenization #llama #nvidia #ai #rtx #gpu #gaming #gpublock #nvidiagtx
969 views2 weeks ago
YouTubeAmit_Chopra_assruc
3:33
Vector Quantization Techniques | Qdrant Multi-Vector Search
99 views1 month ago
YouTubeQdrant Vector Search
13:42
From 15GB to 4.7GB: Quantizing AI Models Locally
7.7K views1 month ago
YouTubeNeuralNine
15:14
Why Inference is hard..
232 views3 weeks ago
YouTubeCaleb Writes Code
0:45
Quantization Explained: How LLMs Get Smaller and Faster
88 views1 month ago
YouTubeDev Alpha Lab
2:36
I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x.All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework.On a 4-token prompt with 252 generated tokens:- Original: 0.76 tok/s- KV cache fp32: 27.21 tok/s- KV cache int8 (quantized): 27.29 tok/sTry it out yourself here: https://t.co/kFS9Z0fs4hIn practice:- KV caching gave us about a 35x end-to-end speedup- INT8 KV cache kept roughly the same speed as fp32 but cut KV cac
48.8K views3 weeks ago
x.comReese Chong
#vlsi #semiconductors #deeplearning #tensorrt #nvidia #aiinfrastructure | Avecas
1 week ago
linkedin.com
1:27
Getting Started with NVIDIA TensorRT
31.6K viewsJul 20, 2021
YouTubeNVIDIA Developer
41:59
Sampling Theorem Quantization and Binary Coding
7.1K viewsApr 11, 2021
YouTubeEngineering with Bingabr
19:56
Faster and Lighter Model Inference with ONNX Runtime from Cloud to Client
3.5K viewsOct 16, 2020
YouTubeMicrosoft Developer
9:58
SmoothQuant
4.4K viewsOct 25, 2023
YouTubeMIT HAN Lab
14:54
TensorRT Overview
45.2K viewsNov 22, 2021
YouTubeAhmad Bazzi
1:08
NVIDIA TensorRT: High Performance Deep Learning Inference
15.7K viewsJul 20, 2021
YouTubeNVIDIA Developer
12:10
Optimize Your AI - Quantization Explained
465.1K viewsDec 28, 2024
YouTubeMatt Williams
1:01
Towards Unified INT8 Training for Convolutional Neural Network
803 viewsJul 17, 2020
YouTubeComputerVisionFoundation Videos
30:35
Inside TensorFlow: Quantization aware training
16.3K viewsJul 23, 2020
YouTubeTensorFlow
1:01:20
tinyML Talks: A Practical Guide to Neural Network Quantization
29.7K viewsSep 30, 2021
YouTubeEDGE AI FOUNDATION
2:51
Quantizing a Deep Learning Network in MATLAB
1.7K viewsJun 15, 2020
YouTubeMATLAB
13:04
Quantization in Deep Learning (LLMs)
11.7K viewsSep 22, 2023
YouTubeAI Bites
36:28
Inference Optimization with NVIDIA TensorRT
17.1K viewsApr 18, 2022
YouTubeNCSAatIllinois
6:42
Linear Quantization Formula | Quantization | TensorTeach
460 viewsNov 20, 2024
YouTubeTensorTeach
18:58
From FP32 to INT8: Post-Training Quantization Explained in PyTorch
928 views6 months ago
YouTubeMLWorks
22:53
Understanding int8 neural network quantization
4.6K viewsJan 28, 2024
YouTubeOscar Savolainen
30:14
LLM Quantization Explained: GPTQ, AWQ, QLoRA, GGUF and More
1.2K views2 months ago
YouTubeTales Of Tensors
See more