SuryanshSinha-suryanshsinha/medical-slm-from-scratch

Building a 92M parameter biomedical language model from scratch in PyTorch — custom BPE tokenizer, decoder-only transformer (RoPE, GQA, SwiGLU, RMSNorm), LoRA fine-tuning on MedQA, and local deployment via FastAPI.

16
/ 100
Experimental
No License No Package No Dependents
Maintenance 13 / 25
Adoption 2 / 25
Maturity 1 / 25
Community 0 / 25

How are scores calculated?

Stars

2

Forks

—

Language

Python

License

—

Last pushed

Apr 05, 2026

Commits (30d)

0

GitHub

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/transformers/SuryanshSinha-suryanshsinha/medical-slm-from-scratch"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

Higher-rated alternatives

Dao-AILab/flash-attention

Fast and memory-efficient exact attention

86

wuwangzhang1216/abliterix

Fully automatic censorship removal for language models. LoRA abliteration + Optuna TPE optimization.

49

lucidrains/deep-cross-attention

Implementation of the proposed DeepCrossAttention by Heddes et al at Google research, in Pytorch

46

modelscope/mcore-bridge

MCore-Bridge: Providing Megatron-Core model definitions for state-of-the-art large models and...

40

assembly-automation-hub/repo-governance

⚙️ Reusable GitHub repository governance kit: CI/CD workflows, CodeQL SAST, Dependabot...

36

Explore Transformer Models

All categories Trending Transformer directory Insights