Bajándole todos los minutos posibles al CI del backend con mas de 1000 tests
The article discusses the optimization of backend continuous integration (CI) processes using over 1000 tests. It highlights the challenges faced in identifying bottlenecks and the eventual realization that the main issue was not what was initially optimized. The author shares insights on various strategies implemented to improve CI times, including the use of alternative package managers and database optimizations.
- ▪The backend is built with FastAPI and includes 1826 tests deployed via GitHub Actions to AWS ECS Fargate.
- ▪Initial CI processes utilized traditional parallelism with two shards via pytest-split.
- ▪The author implemented the 'uv' package manager, which significantly reduced installation times from approximately 60 seconds with pip to around 5.5 seconds.
Opening excerpt (first ~120 words) tap to expand
try { if(localStorage) { let currentUser = localStorage.getItem('current_user'); if (currentUser) { currentUser = JSON.parse(currentUser); if (currentUser.id === 1636392) { document.getElementById('article-show-container').classList.add('current-user-is-article-author'); } } } } catch (e) { console.error(e); } Franchesco Romero for AWS Community Builders Posted on May 26 Bajándole todos los minutos posibles al CI del backend con mas de 1000 tests #githubactions #ci #python Esta es la historia de muchas horas de trabajo en un hoyo continuo tratando de elegir optimizaciones entre package managers, caches paralelismo, optimizaciones de PostgreSQL, advisory locks, y el golpe de realidad de darme cuenta que el cuello de botella no era nada de lo que había estado optimizando, si no algo que no…
Excerpt limited to ~120 words for fair-use compliance. The full article is at DEV.to (Top).
Discussion
0 commentsMore from DEV.to (Top)
