Running large language models on a single GPU for throughput-oriented scenarios.

Run Mixtral-8x7B models in Colab or consumer desktops

PyTorch native quantization and sparsity for training and inference

A QoE-Oriented Computation Offloading Algorithm based on Deep Reinforcement Learning (DRL) for Mobile Edge Computing (MEC) | This algorithm captures the dynamics of the MEC environment by integrating the Dueling Double Deep Q-Network (D3QN) model with Long Short-Term Memory (LSTM) networks.

ZO2 (Zeroth-Order Offloading): Full Parameter Fine-Tuning 175B LLMs with 18GB GPU Memory

LLM Inference on consumer devices

A framework for IoT devices to offload tasks to the cloud, resulting in efficient computation and decreased cloud costs.

Monero hardware wallet protocol implementation for Trezor, agent

A lightweight framework that enables serverless users to reduce their bills by harvesting non-serverless compute resources such as their VMs, on-premise servers, or personal computers.

Code for paper "Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AI" (MobiCom'22)

Backend.AI Client Library for Python

A Pandas-inspired data analysis project with lazy semantics and query-offloading to SQLite

An Activation Offloading Framework to SSDs for Faster Large Language Model Training

Implementations of some popular approaches for efficient deep learning training and inference

Ferramenta para a criação de ambientes de testes com dispositivos Android

Offloading Resource-Intensive Tasks to Raspberry Pi (or IoT Devices) Using SSH

MEES: Mobility-Aware Energy Efficient Scheduling for Cyber-Physical Systems in Computing Continuum

Running large language models on a single M1/M2 GPU for throughput-oriented scenarios.