Freelance AI Infrastructure Engineer

Scope

Set up and configure a local AI server for a civil engineering firm on a dedicated Windows workstation (92GB GPU VRAM). The stack includes WSL2, Docker, vLLM serving a 70B-parameter model (Qwen2.5-72B or Llama 3.3-70B), and AnythingLLM as a RAG document assistant. Internal network deployment for team access also required.

Deliverables

• vLLM running reliably via Docker Compose with auto-restart
• AnythingLLM connected to local model and indexed with firm documents
• Internal network access configured for team
• Basic documentation of the setup for internal IT reference

Skills required

Linux/WSL2, Docker, NVIDIA CUDA, vLLM or similar LLM serving, RAG pipelines

Estimated effort

2-5 days. Open to hourly or fixed-price proposals.

Note

All work involves sensitive engineering project data — candidate must be comfortable working within a fully on-premises, no-cloud-data-transfer constraint. This position is not eligible for remote work. The candidate must be local to Mason, OH.

Work Location: In person