dev-resources.site

for different kinds of informations.

Streamline Your AWS Asset Inventory with Automated Discovery and Reporting

Published at

6/3/2024

Categories

4 categories in total

Author

9 person written this

praveenha

open

Streamline Your AWS Asset Inventory with Automated Discovery and Reporting

Managing resources across multiple AWS accounts and regions can be a daunting task, especially as your cloud infrastructure grows. To simplify this process, I have developed an AWS Inventory Discovery tool that scans all your AWS accounts across all regions and compiles a comprehensive, searchable HTML report. In this blog, I'll walk you through the features, benefits, and technical details of this solution.

Introduction
As organizations scale their use of AWS, keeping track of resources scattered across various accounts and regions becomes increasingly challenging. Manual inventory management is not only time-consuming but also prone to errors. This is where the AWS Inventory Discovery tool comes in.

Features
Comprehensive Scanning: The tool scans all your AWS accounts across all regions, ensuring no resource is left unaccounted for.
Detailed Reporting: Generates a detailed HTML report that lists all discovered resources, including their ARNs and regions.
Searchable Interface: The HTML report includes a search functionality, making it easy to find specific resources quickly.
Visual Representation: Includes pie charts to provide a visual summary of your resources, helping you quickly understand the distribution of resources across various dimensions.
User-Friendly: The generated report is easy to navigate, allowing users to drill down into resource details with just a few clicks.
Benefits
Improved Visibility: Gain a clear overview of all your AWS resources across accounts and regions.
Time Savings: Automate the discovery process, freeing up time for your team to focus on more critical tasks.
Error Reduction: Minimize the risk of overlooking resources or making mistakes in manual inventories.
Enhanced Security: Quickly identify and manage resources to ensure compliance with security policies and best practices.
How It Works
Prerequisites
Before you begin, ensure you have the following set up:

AWS CLI: Install the AWS Command Line Interface (CLI).

pip install awscli
AWS Credentials: Export your AWS profile or access keys and secrets, and specify the region.
export AWS_PROFILE=your-aws-profile

export AWS_ACCESS_KEY_ID=your-access-key-id
export AWS_SECRET_ACCESS_KEY=your-secret-access-key
export AWS_DEFAULT_REGION=your-default-region`

#!/bin/bash

AccountID="<YOUR_AWS_ACCOUNT>"
regions=$(aws ec2 describe-regions --query "Regions[].RegionName" --output text)
#regions=(ap-south-1 eu-north-1  eu-west-3  eu-west-1  ap-northeast-3  ap-northeast-2  ap-northeast-1  ca-central-1  sa-east-1   ap-southeast-1  ap-southeast-2  eu-central-1  us-east-1  us-east-2  us-west-1  us-west-2 )

# Initialize arrays to store data
declare -a summary_data=()
declare -a detailed_data=()

# Fetch resources from all regions
for region in $regions; do
    # Fetch resource ARNs and process them
    data=$(aws --region $region resourcegroupstaggingapi get-resources | jq -r '.ResourceTagMappingList[].ResourceARN' | \
    awk -v region=$region -F '[:/]' '
        {
            resourceType = "unknown"
            if ($3 == "s3") {
                resourceType = "bucket"
            } else if ($3 == "sns") {
                resourceType = "topic"
            } else {
                resourceType = $6  
            }

            # Construct a unique key for each service-resource pair
            pair = $3 ":" resourceType
            count[pair]++
        }
        END {
            for (pair in count) {
                split(pair, s, ":")  # Split the pair back into service and resource
                service = s[1]
                resource = s[2]
                print "{\"service\":\"" service "\", \"resource\":\"" resource "\", \"count\":" count[pair] "}"
            }
        }
    ')

    # Append to summary_data
    summary_data+=($data)

    # Fetch detailed resource information
    detailed_info=$(aws resourcegroupstaggingapi get-resources --region $region | jq -r --arg region $region '.ResourceTagMappingList[] | {Resource_Arn: .ResourceARN, Region: ($region // "Global"), Detailed_Service: (.ResourceARN | split(":")[2])}')

    # Append to detailed_data
    detailed_data+=($detailed_info)
done

# Process summary data
summary_json=$(printf "%s\n" "${summary_data[@]}" | jq -s 'group_by(.service, .resource) | map({service: .[0].service, resource: .[0].resource, count: map(.count) | add})')
total_resources=$(echo "$summary_json" | jq 'map(.count) | add')

# Process detailed data
detailed_json=$(printf "%s\n" "${detailed_data[@]}" | jq -s .)

# Generate HTML
cat <<EOF > index.html
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>AWS Resources</title>
    <style>
        body {
            font-family: Arial, sans-serif;
            background-color: #f4f4f9;
            margin: 0;
            padding: 20px;
        }
        h1 {
            color: #333;
        }
        #searchBox, #searchBox2 {
            width: 100%;
            padding: 10px;
            margin-bottom: 20px;
            box-sizing: border-box;
            font-size: 16px;
        }
        table {
            width: 100%;
            border-collapse: collapse;
        }
        th, td {
            border: 1px solid #ddd;
            padding: 8px;
            text-align: left;
        }
        th {
            background-color: #4CAF50;
            color: white;
        }
        tr:nth-child(even) {
            background-color: #f2f2f2;
        }
        tr:hover {
            background-color: #ddd;
        }
        .total-count {
            font-size: 18px;
            margin: 20px 0;
        }
        #chartContainer {
            width: 50%;
            margin: 0 auto;
        }
    </style>
</head>
<body>
    <h1>AWS Resources for Account ID: $AccountID</h1>
    <div class="total-count">Total Resources: $total_resources</div>
    <input type="text" id="searchBox" onkeyup="filterTable()" placeholder="Search for services..">
    <table id="resourcesTable">
        <thead>
            <tr>
                <th>Service</th>
                <th>Resource</th>
                <th>Count</th>
            </tr>
        </thead>
        <tbody>
        </tbody>
    </table>

    <h1>Detailed AWS Resources</h1>
    <input type="text" id="searchBox2" onkeyup="filterTable2()" placeholder="Search for resources..">
    <table id="detailedResourcesTable">
        <thead>
            <tr>
                <th>Resource ARN</th>
                <th>Region</th>
                <th>Detailed Service</th>
            </tr>
        </thead>
        <tbody>
        </tbody>
    </table>

    <div id="chartContainer">
        <canvas id="resourceChart"></canvas>
    </div>

    <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
    <script>
        const data = $summary_json;
        const resourceData = $detailed_json;

        function loadTableData(data) {
            const tableBody = document.getElementById('resourcesTable').getElementsByTagName('tbody')[0];
            tableBody.innerHTML = "";

            data.forEach(item => {
                let row = tableBody.insertRow();
                let cellService = row.insertCell(0);
                let cellResource = row.insertCell(1);
                let cellCount = row.insertCell(2);

                cellService.innerHTML = item.service;
                cellResource.innerHTML = item.resource;
                cellCount.innerHTML = item.count;
            });

            updateChart(data);
        }

        function loadDetailedTableData(resourceData) {
            const tableBody = document.getElementById('detailedResourcesTable').getElementsByTagName('tbody')[0];
            tableBody.innerHTML = "";

            resourceData.forEach(item => {
                let row = tableBody.insertRow();
                let cellArn = row.insertCell(0);
                let cellRegion = row.insertCell(1);
                let cellService = row.insertCell(2);

                cellArn.innerHTML = item.Resource_Arn;
                cellRegion.innerHTML = item.Region;
                cellService.innerHTML = item.Detailed_Service;
            });
        }

        function filterTable() {
            const searchValue = document.getElementById('searchBox').value.toLowerCase();
            const filteredData = data.filter(item => item.service && item.service.toLowerCase().includes(searchValue));
            loadTableData(filteredData);
        }

        function filterTable2() {
            const searchValue = document.getElementById('searchBox2').value.toLowerCase();
            const filteredResourceData = resourceData.filter(item => 
                item.Resource_Arn.toLowerCase().includes(searchValue) ||
                item.Region.toLowerCase().includes(searchValue) ||
                item.Detailed_Service.toLowerCase().includes(searchValue)
            );
            loadDetailedTableData(filteredResourceData);
        }

        function updateChart(data) {
            const services = data.reduce((acc, item) => {
                acc[item.service] = (acc[item.service] || 0) + item.count;
                return acc;
            }, {});

            const labels = Object.keys(services);
            const counts = Object.values(services);

            const ctx = document.getElementById('resourceChart').getContext('2d');
            new Chart(ctx, {
                type: 'pie',
                data: {
                    labels: labels,
                    datasets: [{
                        data: counts,
                        backgroundColor: [
                            '#FF6384', '#36A2EB', '#FFCE56', '#4CAF50', '#FF9800', '#9C27B0', '#00BCD4'
                        ]
                    }]
                },
                options: {
                    responsive: true,
                    plugins: {
                        legend: {
                            position: 'top',
                        },
                        title: {
                            display: true,
                            text: 'Resource Distribution by Service'
                        }
                    }
                }
            });
        }

        document.addEventListener('DOMContentLoaded', () => {
            loadTableData(data);
            loadDetailedTableData(resourceData);
        });
    </script>
</body>
</html>
EOF

inventory Article's

27 articles in total