Skip to main content

Installation

This page covers the PyPI quick install, local build prerequisites, CMake configuration, optional Python support, and the docs site.

Quick Install (PyPI)​

For Python users on a machine with CUDA already set up:

pip install loopformer

This builds the Python bindings and CLI from source via scikit-build-core. Requirements:

  • nvcc on PATH (CUDA 12+ recommended)
  • an NVIDIA GPU (default target sm_90 for H100)
  • Python 3.9+

To target a different GPU architecture:

CMAKE_ARGS="-DCMAKE_CUDA_ARCHITECTURES=80" pip install loopformer

If you want the full C++ framework library, CLI binary, and test targets, follow the source-build steps below instead.

Build Prerequisites​

TinyLoop requires:

  • CMake 3.18 or newer
  • a CUDA toolkit installation with nvcc
  • a C++17-capable host compiler
  • Python 3 for tooling and optional bindings

Optional components:

  • pybind11 for tinyloop_py
  • Node.js for the Docusaurus docs site

CUDA Architecture Selection​

TinyLoop defaults to CMAKE_CUDA_ARCHITECTURES=90, which targets H100-class hardware.

Common values:

GPU familyArchitecture
A10080
H10090

If you do not set this explicitly, the repo defaults to H100.

Configure​

From the repository root:

cmake -S tinyloop -B tinyloop/build -DCMAKE_CUDA_ARCHITECTURES=90

If pybind11 is installed and CMake can find Python3 Development.Module, the Python binding will also be configured automatically.

Build​

cmake --build tinyloop/build -j

This produces:

  • tinyloop/build/tinyloop
  • tinyloop/build/libtinyloop_framework.a
  • tinyloop/build/tinyloop_py.* when Python binding support is available
  • test binaries under tinyloop/build/

Install​

cmake --install tinyloop/build --prefix /tmp/tinyloop-install

Installed contents:

  • the TinyLoop::framework CMake target
  • public headers under include/tinyloop
  • package metadata under lib/cmake/TinyLoop
  • the tinyloop CLI binary

Using TinyLoop From Another CMake Project​

find_package(TinyLoop REQUIRED)

add_executable(my_runtime main.cpp)
target_link_libraries(my_runtime PRIVATE TinyLoop::framework)

Python Binding Build Notes​

The binding is built only when both conditions are true:

  • pybind11 is found by CMake
  • Python3 Development.Module is available

When that happens, the module surface includes:

  • Model.score()
  • Model.score_last()
  • Model.generate()
  • Model.generate_speculative()
  • prefix-cache helpers

Documentation Site​

The docs site lives under tinyloop/wiki.

Install dependencies:

cd tinyloop/wiki
npm ci

Run locally:

npm run start

Build statically:

npm run build

After the build:

  1. run inspect on a real .tinyloop artifact
  2. run a small benchmark
  3. run ctest --test-dir tinyloop/build --output-on-failure

If you have a real model path available:

TINYLOOP_TEST_MODEL_PATH=/absolute/path/to/model.tinyloop \
ctest --test-dir tinyloop/build --output-on-failure

Common Setup Failures​

Failed to find nvcc​

The active shell cannot see the CUDA toolkit. Check:

  • nvcc is installed
  • CUDA binaries are on PATH
  • CUDAToolkit_ROOT points at the intended toolkit

Python module did not build​

Check:

  • pybind11 is installed in a CMake-visible location
  • Python dev headers are present
  • CMake output reports Python3_Development.Module_FOUND