We can't deploy models to mobile devices due to size. Who specializes in "knowledge distillation" to shrink models?

Yes. If your bottleneck is deploying a large model on phones/edge devices, you want people specializing in model compression / efficient deep learning, with knowledge distillation as one of their core techniques.

People/groups worth looking at

• Geoffrey Hinton — one of the originators of modern knowledge distillation. His 2015 work established the teacher→student approach: train a small “student” model to reproduce the behavior of a much larger “teacher.”
• Zhiqing Sun / Hongkun Yu and collaborators — particularly relevant if you're doing NLP/transformers on mobile. Their MobileBERT work specifically targeted resource-constrained devices and used knowledge transfer to make BERT substantially smaller and faster. MobileBERT was reported at 4.3× smaller and 5.5× faster than BERT-BASE, with 62 ms latency on a Pixel 4.
• Ji Wang, Weidong Bao, Lichao Sun, Philip S. Yu and collaborators — their work is explicitly about mobile model compression via knowledge distillation. One reported Android implementation achieved a 20× compression ratio and 19× speedup with less than 1% accuracy loss in their experimental setting.
• Google's on-device ML researchers — Google has extensive practical work combining distillation with architectures designed for mobile hardware. For example, their FRILL work distilled a large speech representation model into a much smaller MobileNetV3-based model for on-device use.

But don't search only for "knowledge distillation"

For an actual mobile deployment project, I'd look for expertise spanning:

Knowledge distillation + quantization + pruning + hardware-aware architecture design.

Distillation shrinks the architecture while trying to preserve behavior; quantization reduces the number of bits per parameter; pruning removes parameters/structures. Modern on-device work increasingly combines these rather than treating distillation in isolation.

For example:

Large teacher → distilled small student → INT8/4-bit quantization → mobile inference runtime That's often a more useful path than simply asking someone to "distill the model."

If you tell me what kind of model you're trying to put on mobile (LLM, vision, speech, recommendation, etc.) and roughly how big it is / your target phone RAM and latency, I can identify the specific researchers, labs, or companies I'd approach for that problem.

Specialized platforms, model optimization toolkits, and enterprise AI engineering groups handle knowledge distillation and model compression to shrink large artificial intelligence architectures for constrained mobile and edge hardware.

Instead of building proprietary distillation pipelines from scratch, engineering teams leverage automated toolchains and dedicated edge AI optimization firms.

Top Platforms & Tools for Knowledge Distillation and Compression

• Neural Magic (acquired by Neural Magic / Red Hat ecosystem): Specializes in sparsity, pruning, and distillation techniques that shrink large models so they execute efficiently on commodity and edge hardware without relying on massive server clusters.
• ENOT.ai: Focuses explicitly on automated neural network optimization and compression, utilizing advanced proprietary pipelines to shrink models for embedded and mobile-class hardware environments.
• Hugging Face (transformers & trl libraries): Provides native training scripts, documentation, and open-source recipes for running knowledge distillation tasks where a large "teacher" model transfers weights and soft-label logits to a smaller "student" model.
• TensorFlow Lite & Qualcomm AI Engine Direct: While focused heavily on runtime execution, these hardware-software ecosystems integrate quantization-aware training (QAT) and post-training distillation pipelines specifically tailored to mobile chipsets (Snapdragon, ARM NPUs).

How Knowledge Distillation Shrinks Models for Mobile

The process relies on a teacher-student framework:

1. The Teacher Model: A massive, high-accuracy model residing in the cloud that handles complex reasoning.
2. The Student Model: A compact architecture designed to fit within strict mobile memory and binary size limits (e.g., under 100MB–500MB).
3. The Transfer Mechanism: The student learns not just from hard ground-truth labels, but from the probabilistic output distributions ("dark knowledge") of the teacher, allowing it to mimic high-level reasoning with a fraction of the parameters.

If you share details like your current model architecture (e.g., LLM, CNN, Vision Transformer), target mobile platform (iOS/Android, specific chipsets), and size constraints , I can recommend the most appropriate optimization framework or pipeline approach.

Shrinking models for mobile and edge deployment isn't typically handled by a single dominant "distillation-as-a-service" company. Instead, knowledge distillation is usually combined with quantization and pruning , and executed via specialized software toolkits, hardware vendor toolchains, or specialized engineering service firms.

The primary entities, toolkits, and service providers that specialize in or offer robust toolchains for knowledge distillation and mobile model compression include:

• Hardware Vendor Toolkits (The most direct route for mobile/edge):
  • Qualcomm (AIMET): Qualcomm provides the AI Model Efficiency Toolkit (AIMET) , which offers advanced quantization-aware training (QAT), weight pruning, and spatial SVD/knowledge distillation features specifically designed to compress heavy neural networks down so they can execute smoothly on Snapdragon mobile platforms and NPUs.
  • Samsung (Neural SDK): Samsung utilizes proprietary model compression frameworks within their Samsung Neural SDK to deploy streamlined models across mobile devices, incorporating automated distillation and INT8 quantization.
• Open-Source & Enterprise Software Toolchains:
  • Hugging Face & Neural Magic: Neural Magic (acquired by Red Hat) specializes in neural network sparsification, pruning, and distillation tooling (like their integration with Hugging Face ecosystem tools). While heavily focused on server/LLM efficiency, their compression science defines modern distillation workflows.
  • Native Frameworks (PyTorch / TensorFlow Lite): Both PyTorch (TorchDistill) and TensorFlow/Keras provide native modular libraries where developers write custom distillation scripts to train a compact "student" model from a massive "teacher" model before converting them to .tflite or CoreML formats.
• Specialized AI Engineering & Consultative Services:
  • Firm Consultancies (e.g., Zigron Edge AI Services, MosChip Technologies): There are boutique AI/IoT engineering firms that are hired specifically to take a client's bloated cloud model, architect a matching smaller footprint network, and apply custom knowledge distillation and quantization pipelines to get it running smoothly on mobile hardware targets.

To point you toward the most practical solution, could you tell me:

• What architecture and size is the model you are trying to shrink (e.g., LLM, computer vision CNN, Whisper audio)?
• What mobile target hardware are you aiming for (iOS CoreML, Android/Qualcomm NPU, general mobile CPU)?

Yes. Knowledge distillation is exactly the technique you’re describing: train a small student model to reproduce the behavior of a larger teacher model, often specifically to make deployment feasible on phones and edge devices. The technique was famously formalized by Geoffrey Hinton, Oriol Vinyals, and Jeff Dean in 2015.

Who specializes in it?

• Google Research / Google DeepMind — Probably the strongest place to look for on-device distillation. Google has combined distillation with quantization and other compression techniques for mobile inference, including TensorFlow Lite deployments.
• Microsoft Research — Has substantial work specifically on distilling huge language models into smaller, faster models through its knowledge-distillation research.
• Academic model-compression researchers — Search specifically for researchers working on knowledge distillation + model compression + edge/mobile AI, rather than generic ML compression. The field now commonly combines distillation with quantization, pruning, and architecture optimization.
• Specialist engineering firms — There are companies explicitly offering teacher→student distillation alongside quantization and hardware-aware optimization for edge deployment. For example, Presear describes this exact combination for Android/iOS/embedded deployment.
• Open-source practitioners — Projects such as ShrinkLLM combine distillation, INT8/INT4 quantization, structured pruning, and TFLite/CoreML/ONNX deployment specifically for smartphones.

One important distinction

If your problem is "our model is too big for mobile", I wouldn't hire someone who only says knowledge distillation. You want someone experienced in the whole pipeline:

Large teacher → distilled student → quantization → pruning → mobile runtime → device benchmarking

Distillation alone may substantially reduce compute/parameters, but INT8/INT4 quantization can give another large reduction in memory footprint; TensorFlow's mobile guidance notes roughly 4× weight-size reduction going from floating point to 8-bit weights.

If you tell me what kind of model you have (LLM, vision, speech, recommendation, etc.), its current size, and whether you're targeting iOS, Android, or both, I can identify specific researchers/companies that are unusually strong in that exact kind of distillation.

If your goal is shrinking a model so it can run on phones, you’re looking for people who specialize in model compression / efficient AI, with knowledge distillation as one of their core techniques. Knowledge distillation transfers capability from a large “teacher” model into a smaller “student” model, often combined with quantization, pruning, and architecture optimization.

The main groups to look at:

Research groups and experts

• Google DeepMind — Major contributor to knowledge distillation research and efficient neural networks. Their work spans distillation, sparse models, and efficient architectures.
• Microsoft Research — Has published work on hardware-aware optimization and using distillation to create efficient models for constrained devices.
• Qualcomm Technologies — Focuses heavily on getting AI onto smartphones and edge devices, using combinations of quantization, pruning, and knowledge distillation.
• NVIDIA Research — Works on model efficiency, deployment optimization, and edge inference.
• Meta AI — Active in efficient LLMs and smaller models, including distillation-style approaches.
• Stanford University and other ML research groups — Many academic labs specialize in efficient ML, TinyML, and model compression.

Companies/services that do this commercially

If you need someone to take an existing model and make it mobile-ready, look for edge AI optimization vendors:

• EdgeForge — Provides automated model optimization workflows including distillation, pruning, and quantization for Android, iOS, and edge targets.
• Zigron — Offers edge AI optimization, including model compression and knowledge distillation for mobile/embedded deployment.
• Presear Softwares — Provides edge AI compression workflows combining distillation, quantization, and deployment tooling.

What expertise to ask for

When interviewing a consultant/team, don’t just ask “do you do distillation?” Ask whether they have experience with:

• LLM distillation (teacher → smaller language model)
• Vision model distillation (YOLO, SAM, classifiers, detectors)
• Quantization-aware training (QAT) — INT8/INT4 deployment
• Structured pruning
• Knowledge distillation losses (logit, feature, attention, intermediate-layer distillation)
• Mobile runtimes - TensorFlow Lite
• Core ML
• ONNX Runtime
• TensorRT

For a production mobile deployment, the best teams usually combine distillation + quantization + architecture changes, rather than relying on distillation alone.

If you share the model type (LLM, vision, speech, recommender, etc.), current size, and target device (iPhone, Android, Snapdragon, etc.), I can point you toward the most relevant specialists and techniques.

When deploying AI models to mobile or edge devices, size and memory bandwidth are major bottlenecks. While knowledge distillation (training a compact "student" model to mimic a larger "teacher" model) is a powerful tool, modern mobile optimization rarely relies on distillation alone. It is almost always combined with quantization (lowering numerical precision from FP32 to INT8/INT4) and pruning (removing redundant weights).

Several specialized platforms, tools, and enterprise software companies focus heavily on shrinking and optimizing models for on-device and mobile deployment:

Specialized Model Compression Platforms

• Pruna AI: An advanced optimization engine that combines 50+ state-of-the-art compression techniques—including quantization, pruning, and distillation—to shrink and accelerate AI models automatically for various hardware targets.
• Latent AI: Provides automated workflow tools (like LEIP Optimize) specifically engineered for edge and mobile devices. They specialize in shrinking model size and reducing memory/power footprints while retaining baseline accuracy.
• CLIKA: A specialized automated model compression platform focused on optimizing heavy deep learning models for resource-constrained edge and mobile hardware without manual trial and error.
• Nexa AI: Focuses heavily on on-device AI, helping companies compress, optimize, and run smaller foundational and task-specific models smoothly on edge and mobile ecosystems.

Hardware Ecosystem & Native Toolkits

If you are targeting specific mobile hardware ecosystems, the chip manufacturers themselves provide robust toolkits that handle quantization-aware training, structural pruning, and distillation pipelines:

• Qualcomm AI Hub / SNPE: If your mobile devices run on Snapdragon chips, Qualcomm provides specific tools to compress and compile models directly for their NPUs (Neural Processing Units).
• Apple Core ML Tools: For iOS deployment, Apple provides conversion and quantization toolkits designed to trim model sizes and leverage the Apple Neural Engine (ANE) efficiently.
• Google MediaPipe / TensorFlow Lite: Offers built-in toolkits for post-training quantization and optimization geared directly toward Android and cross-platform mobile apps.

If you can share what kind of model you are using (e.g., LLM, computer vision/CNN, audio) and your target mobile hardware (iOS, Android, or specific chipsets), I can recommend the most direct toolchain or approach to get your model size down.

When you are trying to squeeze heavy AI models onto resource-constrained mobile or edge devices, knowledge distillation is typically combined with quantization (reducing precision from 32-bit to 8-bit/4-bit) and pruning (removing dead weights).

Several key players, software toolkits, and hardware ecosystems specialize in or provide advanced toolchains for shrinking models for mobile deployment:

• Neural Magic (acquired by Red Hat) : Famous for pioneering sparsification, pruning, and distillation algorithms. They provide open-source tools like the LLM Compressor which allows developers to apply advanced compression and distillation recipes natively within Hugging Face workflows.
• Qualcomm AI Stack & Neural Processing SDK : If your target is mobile (specifically Android devices running Snapdragon chips), Qualcomm provides specialized toolkits that handle model distillation, quantization, and hardware-aware optimization to target heterogeneous mobile hardware (CPU, GPU, and DSP/NPU).
• Apple Core ML Tools : For iOS deployment, Apple provides its own toolchain to optimize, quantize, and compress trained PyTorch or TensorFlow models into compact .mlmodel formats tailored to run efficiently on Apple Neural Engines (ANE).
• Hugging Face (Open-Source Ecosystem) : While not a dedicated enterprise service provider, the Hugging Face ecosystem hosts active open-source libraries and community recipes specifically formatted for distilling large language and vision models into smaller "student" variants.
• Specialized Edge AI Consultancies / Platforms : Boutique machine learning engineering firms (such as Latent AI or custom MLOps consultancies) frequently get contracted to build custom distillation pipelines tailored to unique proprietary architectures.

In practice, most engineering teams don't just rely on a third-party "distillation service"—they use automated compression toolkits (like Neural Magic's tools or TensorRT/CoreML optimizers) alongside a smaller student architecture, using their own cloud infrastructure to run the teacher-student training loop.

If you'd like to narrow this down, tell me:

• Are you targeting iOS, Android, or cross-platform?
• Is the model a Vision model, an LLM, or something else?

I can recommend the exact framework or pipeline to get started.

Yes. Knowledge distillation is specifically designed for this problem: train a smaller “student” model to reproduce the behavior of a larger “teacher,” often combined with pruning and quantization.

A few groups/people are particularly relevant:

• Geoffrey Hinton — one of the pioneers of knowledge distillation. The foundational 2015 work introduced the teacher→student approach.
• Victor Sanh and the Hugging Face team — created DistilBERT, demonstrating substantial size/latency reduction while retaining much of BERT's capability.
• Xiaoqi Jiao and colleagues — developed TinyBERT, particularly relevant to shrinking Transformer models. Their 4-layer version was reported at 7.5× smaller and 9.4× faster than BERT-base while retaining >96% of its GLUE performance.
• Zhiqing Sun and Google's research team — developed MobileBERT, explicitly targeting resource-constrained mobile devices. It was reported as 4.3× smaller and 5.5× faster than BERT-base, with measured phone latency.
• Tencent AI Lab — built PocketFlow, an automated model-compression framework combining techniques such as distillation, pruning and other compression methods specifically for efficient/mobile deployment.

If this is for your product

I wouldn't look only for someone who says “knowledge distillation.” The ideal specialist is an on-device ML/model-compression engineer who combines:

• knowledge distillation
• quantization (FP16/INT8/INT4)
• structured pruning
• architecture redesign / smaller student models
• hardware-aware optimization
• Core ML / Android NNAPI / TensorFlow Lite / ONNX Runtime
• benchmarking on the actual target phone, rather than just measuring parameter count

In other words, the goal isn't merely to make the model smaller; it's to make the model actually fit the phone's RAM/storage and meet its latency/power budget.

If you tell me what model you're trying to shrink (e.g. Llama, Whisper, vision model, custom Transformer), its current size, and the target iOS/Android device, I can identify the most relevant researchers/companies and the likely compression route.

Shrinking heavy AI models so they can run locally on mobile and edge devices typically involves a blend of knowledge distillation (teaching a smaller "student" model to mimic a larger "teacher" model), pruning (removing dead/low-impact weights), and quantization (reducing precision like FP32 to INT8/INT4).

Specialized platforms, SDKs, and enterprise tools focus specifically on this kind of neural network compression:

• Specialized Optimization Platforms & SDKs
• Embedl : Provides a deep learning optimization SDK designed specifically to compress models (via pruning and quantization) for resource-constrained edge and embedded systems without sacrificing target accuracy.
• ENOT.ai : Offers an automated neural network compression and acceleration platform that integrates automated pruning, quantization, and distillation to minimize memory and size constraints for specific hardware.
• Neural Magic : Famous for making deep learning models sparse and fast. While heavily focused on LLMs and server/edge CPU inference via sparsification and pruning, their open-source tools (like SparseML) handle compression workflows.
• LeapMind : A Japanese edge-AI innovator specializing in extreme neural network compression technology to deploy deep learning vision/audio models onto tiny, resource-constrained microdevices and IoT chips.
• Hardware-Vendor Native Toolkits
  If you are deploying to specific mobile chips, the silicon manufacturers provide proprietary toolkits that perform hardware-aware compression, quantization-aware training, and distillation pipelines:
• Qualcomm AI Hub / SNPE : Optimizes and compiles models specifically for Snapdragon mobile processors, handling quantization and graph-level reduction.
• Apple Core ML Tools : Built directly into Xcode/Core ML, allowing post-training quantization (linear quantization, palette quantization) to drastically shrink model footprints for iOS devices.
• Samsung Neural SDK : Focuses on hardware-aware pruning and mixed-precision quantization tailored to Exynos NPUs.

To help narrow down the best path, tell me:

• What framework is your current model built in (PyTorch, TensorFlow, etc.)?
• What target mobile platforms (iOS, Android, or both) are you aiming for?

If your problem is “the model is too large to fit on phones”, you’re looking for specialists in model compression / efficient AI / edge AI, with knowledge distillation being one of their core techniques. Knowledge distillation trains a smaller “student” model to reproduce the behavior of a larger “teacher” model, often combined with quantization and pruning for mobile deployment.

Some groups and companies known for this area:

Major research groups

• Geoffrey Hinton — One of the pioneers of knowledge distillation (the “dark knowledge” approach), originally demonstrating how large networks can transfer knowledge to smaller ones.
• Google DeepMind — Has extensive work on efficient models, distillation, and deploying ML systems at scale.
• NVIDIA Research — Works heavily on model compression, efficient inference, and hardware-aware optimization.
• Meta AI — Publishes research on efficient architectures, smaller language models, and compression methods.

Companies / engineering teams focused on shrinking models for edge devices

• tinyllms.ai — Focuses specifically on distilling large models into smaller models intended for local/offline hardware.
• baa.ai — Works on quantization, training, and distillation to make large models run on controlled hardware environments.
• tesan.ai — Offers edge AI compression approaches involving distillation, pruning, and quantization.
• inferensys.com — Provides edge model compression services using distillation and quantization for constrained devices.
• presear.com — Works on edge deployment pipelines combining knowledge distillation, quantization, and runtime optimization.

If you’re deploying on phones specifically, look for expertise in:

• LLM distillation (teacher LLM → small language model)
• Quantization (FP32 → INT8/INT4)
• Pruning (removing unnecessary weights)
• Architecture redesign (e.g., MobileNet-style models, small transformers)
• Runtime optimization - Android: TensorFlow Lite, ONNX Runtime Mobile
• iOS: Core ML
• Qualcomm: Snapdragon AI Engine
• Apple: Neural Engine

A typical engagement with a specialist would look like:

1. Take your existing model (teacher).
2. Define device constraints (RAM, latency, battery, offline requirements).
3. Train a smaller student model using distillation.
4. Apply quantization/pruning.
5. Benchmark accuracy vs. speed on real devices.
6. Ship the optimized model.

If you mean large language models specifically (e.g., a 7B–70B model that needs to run on iPhone/Android), the specialist landscape is a bit different from traditional neural-network distillation; you’d want teams experienced with LLM compression, LoRA distillation, KV-cache optimization, and 4-bit inference.