prefill

LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.

A from-scratch Prefill/Decode disaggregation inference engine for LLMs

Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parallel architecture, suitable for MOE model hybrid inference.

Prefills factories with faker method suggestions to increase productivity

LANCache Manager - Self-hosted web dashboard for monitoring Lancache data, Real-time download tracking, bandwidth analytics, cache management, and client monitoring. Docker deployment with Prometheus metrics and Grafana integration.

Multi-GPU prefill acceleration for llama.cpp

Adaptive Disaggregated Inference on a Role-free Fleet!

(WORKS) NSFW/Coding refusals removing local proxy. If ENI is not enough.

Generate pre-filled PO files from POT file, using source PO files in gulp.

NanoPD, a minimalistic implementation of the disaggregated PD serve

Wire the 1CatAI Split-D D256 FlashAttention kernel (fishlikeX/sm70-attn, MIT) into NInfer on Tesla V100 sm_70: +36-41% prefill, TTFT -3min, decode unchanged. Measured data + integration guide. Published by the user with AI assistance.

Apple Neural Engine (ANE/NPU) vs GPU for local LLM inference on Apple Silicon / macOS. Core ML (CoreML), Core AI (CoreAI), MLX and Metal benchmarks: prefill, throughput, latency, memory, thermals, INT4/INT8, W4A16/A8W4 quantization, grouped scales and FP16 arithmetic. Reproducible component tests, compatibility findings, English/Chinese articles.

Generate pre-filled PO files from POT file, using source PO files in NodeJS.

CPU-only LLM inference research: session KV reuse, HF/GGUF backends, reproducible benchmarks, and long-context quality evaluations with raw results.

Measure and visualize why LLM inference is slow: bottleneck analysis, model dissection, KV-cache, GEMM/GEMV, quantization, and memory-bound decoding.

Auto-fill any Google Form with details you've saved once — Chrome extension, Android app, and a read-only API for what a form asks.

Automatically pre-fill course section CRNs or select a template for instant RPI class registration when your time ticket is activated.

React component to create a button/link to prefill Codepen.io with supplied code.

Event-driven benchmark of adaptive batch composition policies for LLM serving, measuring how prefill and decode interference affects TTFT, TPOT, and throughput under different memory pressure regimes.