🚀 Optimizing Storage Costs with Amazon Q Developer:
Building the AWS EFS Analyzer
As a associate system administrator I worked on Redhat Linux servers, including user management, permissions, services, and performance monitoring Automated routine administrative tasks using Bash scripting and cron jobs, reducing manual effort by ~30% I am aws certified sysops administrator and Google Certified Cloud Engineer. Determined to transition my career into cloud architect /Cloud Support role
Hey Techies! 👋
Managing cloud costs is one of the biggest challenges for DevOps and Cloud Engineers today. We often provision storage like Amazon EFS (Elastic File System) and forget to check if we are actually using the data we're paying for. Did you know you can save up to 92% on storage costs just by moving infrequently accessed files to the right tier?
Today, I’m excited to share my submission for the Amazon Q Developer "Quack The Code" Challenge: Crushing the Command Line. I built a tool that automates this entire process!
🛠️ What I Built: The AWS EFS Analyzer
The AWS EFS Analyzer is a powerful Python-based command-line tool designed to help AWS users identify massive cost-saving opportunities. Instead of manually checking thousands of files, this tool recursively scans your EFS mount points and gives you a clear roadmap to savings.
Why is this important?
Amazon EFS offers different storage tiers:
Standard: For frequently accessed data.
Infrequent Access (IA): For data not accessed daily.
Archive: For very rare access.
Manual identification is tedious and error-prone. My tool automates this, categorizing data and projecting monthly savings in seconds!
🧠 The Development Journey with Amazon Q Developer
I didn't build this alone! Amazon Q Developer was my AI pair programmer throughout the process. It helped me move from a basic idea to a production-ready CLI tool.
📝 The Core Prompt
I started with a comprehensive prompt to define the logic:
"Create a Python script that analyzes an Amazon EFS mount point... support parallel processing... categorize files by last access time (7, 14, 30, 60+ days)... and generate HTML/Text reports."
🔄 Iterative Improvements (The "Q" Factor)
The best part of using Amazon Q was the iterative refinement. Here is how we polished the tool:
Handling Permissions: I ran into errors scanning
/proc.Prompt: “Fix the permission errors when scanning system directories like /proc.”
Result: Q implemented robust error handling to skip protected system files without crashing the script.
User Safety: * Prompt: “Add a confirmation prompt that warns about CPU usage.”
- Result: The tool now asks for permission before starting a heavy parallel scan, preventing accidental performance hits on production servers.
Visual Feedback:
Prompt: “Add a clean progress bar with ETA.”
Result: Using the
tqdmlogic, Q helped me implement a real-time progress tracker.
✨ Key Features
✅ Parallel Processing: Handles large file systems efficiently using multi-core processing.
✅ Smart Categorization: Groups files by access age (7 days to 2+ years).
✅ Cost Projection: Calculates exactly how much you’d save by switching to IA or Archive tiers.
✅ Interactive Reports: Generates beautiful HTML reports with visualizations and plain text summaries.
✅ Safety First: Detects symbolic link loops and automatically excludes system directories.
🚀 How to Use the AWS EFS Analyzer
Using the tool is simple. Once you have your EFS mounted, you can run:
Bash
python3 efs_analyzer.py --path /mnt/my-efs --threads 4 --output-dir ./reports
What happens next?
The tool scans your mount point.
It analyzes the
atime(access time) of every file.It generates a report showing your Current Cost vs. Optimized Cost.
Note: Even a small change in Lifecycle Management can save a mid-sized company thousands of dollars per month!
📚 What I Learned
Building this project taught me three major things:
Clear Prompts = Better Code: Amazon Q is incredibly powerful when you give it specific constraints (like avoiding
/devor/sys).Iterative AI Dev: You don't have to get it right the first time. You can build the "skeleton" and then ask Q to "add muscle" (features).
EFS Economics: Understanding the price gap between Standard and Archive tiers is eye-opening.
📦 Code Repository & Demo
You can explore the code, contribute, or fork the project here:
🔗 GitHub Repository: aws-efs-analyzer
🎯 Conclusion
Amazon Q Developer has fundamentally changed my workflow. It allowed me to focus on the logic and impact of the tool while it handled the boilerplate and edge-case debugging.
If you're looking to crush your AWS bills, give the EFS Analyzer a try!