CrewAI ReadMe Generator, a fully customised Crew

.
├── agents
│   ├── analyse_files_agent.py
│   ├── __init__.py
│   ├── order_files_agent.py
│   └── readme_generator_agent.py
├── config.json
├── main.py
├── requirements.txt
├── tasks
│   ├── analyse_files_task.py
│   ├── generate_readme_task.py
│   ├── __init__.py
│   └── order_files_task.py
├── tools
│   ├── analyse_files.py
│   ├── __init__.py
│   ├── list_files.py
│   ├── search.py
│   └── write_readme.py
└── utils
    ├── file_utils.py
    └── __init__.py

from crewai import Task
from agents.analyse_files_agent import AnalyseFilesAgent

analyse_files_task = Task(
  description="""Analyse the content of specific files in the project to extract key information and insights.
  Files should be analysed one at a time ot prevent hitting LLM limits""",
  expected_output="""For each file, you should output the name and a brief explanation of any functions or similar constructs in the file.
                  Check for any dependencies that may be present, including references to elements in other files in the project.
                  Include a summary of the file as a whole.""",
  agent=AnalyseFilesAgent
)

import os
from crewai import Agent
from dotenv import load_dotenv
from langchain_anthropic import ChatAnthropic
from tools.list_files import list_files

load_dotenv()

ClaudeHaiku = ChatAnthropic(
  model="claude-3-haiku-20240307"
)
ClaudeSonnet = ChatAnthropic(
  model="claude-3-sonnet-20240229"
)

ANTHROPIC_API_KEY = os.environ.get('ANTHROPIC_API_KEY')

OrderFilesAgent = Agent(
  role='File Prioritise Agent',
  goal='Order files based on importance and relevance.',
  backstory="""You are a file prioritisation expert with a keen eye for detail.
    Your task is to order the files in the project directory based on their importance and relevance.
    You should rely solely on the `list_files` tool to list the project files out, based on their name and file type, you should recognise the type of project and order them accordingly.
    Your output should be in the same format as the input list, including all files and their paths, if that was what was passed to you.
    Pass this information on to the `AnalyseFilesAgent` for further analysis.""",
  verbose=True,
  allow_delegation=True,
  llm=ClaudeHaiku,
  max_iter=5,
  memory=True,
  tools=[list_files]
)

backstory="""A specialist in file analysis, adept at extracting key insights from code and documentation.
    Your role is to provide valuable information from the project files for the README compilation.
    Only use the search function if you need to look up additional information.
    You should be particularly interested in and dependencies between files and any complex data manipulation that requires explanation.
    Your output should be a summary of key information extracted from the project files.
    You should look at one file at a time, commit the output of the `analyse_files_task` to memory before moving on to the next file.
    To use the analyse_files tool, you should call it with `analyse_files(file_name)` where `file_name` is the name of the file you want to analyse.
    Once collated, pass this information on to the `ReadMeGeneratorAgent` for README generation.""",

import json
import time
from crewai_tools import tool, FileReadTool
from pathlib import Path


@tool('analyse_files')
def analyse_files(file_name):
  """Analyse a single file in the project by fetching the base path from config and appending it to the filename."""
  try:
    # Load the base path from the config file
    with open('config.json', 'r') as config_file:
      config = json.load(config_file)
    base_path = config.get('last_path', '')

    # Construct the full path to the file
    full_path = Path(base_path) / file_name

    # Introduce a delay to manage rate limits
    time.sleep(8)

    # Perform the file analysis
    file_tool = FileReadTool(file_path=str(full_path))
    content = file_tool.run()
    return {file_name: content}
  except FileNotFoundError:
    print(f"Error: 'config.json' not found or '{file_name}' does not exist.")
    return {file_name: None}
  except json.JSONDecodeError:
    print("Error: JSON decoding failed. Check the format of 'config.json'.")
    return {file_name: None}
  except Exception as e:
    print(f"Error analysing file {full_path}: {e}")
    return {file_name: None}

# Project README

## Overview

This project is managed using Terraform and deploys resources across multiple AWS regions, including `eu-west-1` (Dublin), `us-east-1` (Virginia), and `ap-south-1` (Mumbai).

## Key Components

### Terraform Configuration

- The `main.tf` file sets the required Terraform version (1.7.5 or higher) and configures multiple AWS provider blocks, each with a different region and profile.

### Variables and Local Values

- The `variables.tf` file defines a single variable, `environment`, with a default value of `"dev"`. It also sets up a set of global tags in the `locals` block, including `Environment`, `Department`, and `Owner`.

### EC2 Instances

- The `ec2.tf` file is responsible for creating EC2 instances in the different regions. It looks up the latest Ubuntu 22.04 ARM64 AMI in each region and uses a `for_each` loop to create the instances based on the `local.ec2_map` variable.
- The instances are configured with various attributes, such as the AMI, instance type, subnet, IAM instance profile, user data, key pair, and root block device settings.
- The instances are tagged with the `local.global_tags` and additional tags, such as `Name`, `ShutdownTime`, and `StartTime`.

### Virtual Private Clouds (VPCs)

- The `vpc.tf` file manages the creation of VPCs in the different regions. It uses a `for_each` loop to create the VPCs based on the `local.vpc[var.environment]` variable.
- Each VPC is configured with a CIDR block, DNS support, and DNS hostnames.
- The VPCs are tagged with the `local.global_tags` and a `Name` tag.

### Public Subnets and Route Tables

- The `subnets.tf` file creates public subnets and route tables in the different regions.
- It uses a `for_each` loop to create the public subnets based on the `local.subnet_map` variable, with each subnet configured with a VPC ID, CIDR block, and availability zone.
- The file also creates public route tables and associates them with the public subnets.

## Conclusion

This Terraform project manages the deployment of resources, including EC2 instances, VPCs, and public subnets, across multiple AWS regions. The configuration is driven by various variables and local values, providing a flexible and reusable solution.

The project's unusual or complex aspects include:

1. **Multi-region Deployment**: The project deploys resources across three different AWS regions: Dublin, Virginia, and Mumbai. This requires careful configuration of the AWS providers, VPCs, and subnets to ensure consistent and reliable infrastructure.

2. **Dynamic EC2 Instance Creation**: The `ec2.tf` file uses a `for_each` loop to dynamically create EC2 instances based on the `local.ec2_map` variable. This allows for easy scaling and modification of the EC2 infrastructure without having to manually update the Terraform configuration.

3. **Dynamic VPC and Subnet Creation**: Similar to the EC2 instances, the `vpc.tf` and `subnets.tf` files use `for_each` loops to dynamically create VPCs and subnets based on the `local.vpc` and `local.subnet_map` variables. This adds flexibility and reusability to the infrastructure deployment.

4. **AMI Lookup**: The project uses the `aws_ami` data source to lookup the latest Ubuntu 22.04 ARM64 AMI in each region. This ensures that the EC2 instances are always provisioned with the most up-to-date base image.

Overall, this Terraform project demonstrates a robust and flexible multi-region infrastructure deployment, with several advanced features that may require additional explanation or documentation for users unfamiliar with the project.