Logo

dev-resources.site

for different kinds of informations.

Golang Web API: Project configuration management using Viper

Published at
10/15/2023
Categories
go
programming
gin
webdev
Author
naeemaei
Main Article
https://dev.to/naeemaei/golang-web-api-course-project-configuration-management-using-viper-3fkh
Categories
4 categories in total
go
open
programming
open
gin
open
webdev
open
Author
8 person written this
naeemaei
open
Golang Web API: Project configuration management using Viper

In this section, I create separate config files (yaml) for the development environment and docker environment, then make a config package to read configuration from yaml files.
To see this session project you can visit Session 4 GitHub repo URL.

Create config files

In the first step, we need to make three config files for the docker, production, and development environments in src/config path:

1. config-development.yml 

server:
  internalPort: 5005
  externalPort: 5005
  runMode: debug
logger:
  filePath: ../logs/
  encoding: json
  level: debug
  logger: zap
cors:
  allowOrigins: "*"
postgres:
  host: localhost
  port: 5432
  user: postgres
  password: admin
  dbName: car_sale_db
  sslMode: disable
  maxIdleConns: 15
  maxOpenConns: 100
  connMaxLifetime: 5
redis:
  host: localhost
  port: 6379
  password: password
  db: 0
  dialTimeout: 5
  readTimeout: 5
  writeTimeout: 5
  poolSize: 10
  poolTimeout: 15
  idleCheckFrequency: 500
password:
  includeChars: true
  includeDigits: true
  minLength: 6
  maxLength: 64
  includeUppercase: true
  includeLowercase: true
otp:
  expireTime: 120
  digits: 6
  limiter: 100
jwt:
  secret: "mySecretKey"
  refreshSecret: "mySecretKey"
  accessTokenExpireDuration: 1440
  refreshTokenExpireDuration: 60
Enter fullscreen mode Exit fullscreen mode

2. config-docker.yml 

server:
  internalPort: 5000
  externalPort: 0
  runMode: release
logger:
  filePath: /app/logs/
  encoding: json
  level: debug
  logger: zap
cors:
  allowOrigins: "*"
postgres:
  host: postgres_container
  port: 5432
  user: postgres
  password: admin
  dbName: car_sale_db
  sslMode: disable
  maxIdleConns: 15
  maxOpenConns: 100
  connMaxLifetime: 5
redis:
  host: redis_container
  port: 6379
  password: password
  db: 0
  dialTimeout: 5
  readTimeout: 5
  writeTimeout: 5
  poolSize: 10
  poolTimeout: 15
  idleCheckFrequency: 500
password:
  includeChars: true
  includeDigits: true
  minLength: 6
  maxLength: 64
  includeUppercase: true
  includeLowercase: true
otp:
  expireTime: 120
  digits: 6
  limiter: 100
jwt:
  secret: "mySecretKey"
  refreshSecret: "mySecretKey"
  accessTokenExpireDuration: 60
  refreshTokenExpireDuration: 60
Enter fullscreen mode Exit fullscreen mode

3. config-production.yml 

server:
  internalPort: 5010
  externalPort: 5010
  runMode: release
logger:
  filePath: logs/
  encoding: json
  level: debug
  logger: zap
cors:
  allowOrigins: "*"
postgres:
  host: localhost
  port: 5432
  user: postgres
  password: admin
  dbName: car_sale_db
  sslMode: disable
  maxIdleConns: 15
  maxOpenConns: 100
  connMaxLifetime: 5
redis:
  host: localhost
  port: 6379
  password: password
  db: 0
  dialTimeout: 5
  readTimeout: 5
  writeTimeout: 5
  poolSize: 10
  poolTimeout: 15
  idleCheckFrequency: 500
password:
  includeChars: true
  includeDigits: true
  minLength: 6
  maxLength: 64
  includeUppercase: true
  includeLowercase: true
otp:
  expireTime: 120
  digits: 6
  limiter: 100
jwt:
  secret: "mySecretKey"
  refreshSecret: "mySecretKey"
  accessTokenExpireDuration: 1440
  refreshTokenExpireDuration: 60
Enter fullscreen mode Exit fullscreen mode

4. Config structure in go

All configuration of the project must be placed in config-{env}.yml files. For now, server, logger, postgres, cors, and Redis configurations are placed in this file and will be completed later.
Now it's time to start making config.go file to the central point of the project environment the configuration management. this file is placed in src/config directory. First, we need to create structs corresponding to the config sections:

type Config struct {
    Server   ServerConfig
    Postgres PostgresConfig
    Redis    RedisConfig
    Password PasswordConfig
    Cors     CorsConfig
    Logger   LoggerConfig
    Otp      OtpConfig
    JWT      JWTConfig
}

type ServerConfig struct {
    InternalPort    string
    ExternalPort    string
    RunMode string
}

type LoggerConfig struct {
    FilePath string
    Encoding string
    Level    string
    Logger   string
}

type PostgresConfig struct {
    Host            string
    Port            string
    User            string
    Password        string
    DbName          string
    SSLMode         string
    MaxIdleConns    int
    MaxOpenConns    int
    ConnMaxLifetime time.Duration
}

type RedisConfig struct {
    Host               string
    Port               string
    Password           string
    Db                 string
    DialTimeout        time.Duration
    ReadTimeout        time.Duration
    WriteTimeout       time.Duration
    IdleCheckFrequency time.Duration
    PoolSize           int
    PoolTimeout        time.Duration
}

type PasswordConfig struct {
    IncludeChars     bool
    IncludeDigits    bool
    MinLength        int
    MaxLength        int
    IncludeUppercase bool
    IncludeLowercase bool
}

type CorsConfig struct {
    AllowOrigins string
}

type OtpConfig struct {
    ExpireTime time.Duration
    Digits     int
    Limiter    time.Duration
}

type JWTConfig struct {
    AccessTokenExpireDuration  time.Duration
    RefreshTokenExpireDuration time.Duration
    Secret                     string
    RefreshSecret              string
}
Enter fullscreen mode Exit fullscreen mode

5. Config functions

In this project, we use viper package to manage configurations. So first we should install the Viper package go get -u github.com/spf13/viper. Then run go mod tidy command to ensure that all imports are satisfied.
At this stage, we need some functions

5.1 Get config file path

this function returns a file path based environment.

func getConfigPath(env string) string {
    if env == "docker" {
        return "/app/config/config-docker"
    } else if env == "production" {
        return "/config/config-production"
    } else {
        return "/config/config-development"
    }
}
Enter fullscreen mode Exit fullscreen mode
5.2 Load config file with Viper

After the config file was found, we needed to load the config file with Viper based specified file type.

func LoadConfig(filename string, fileType string) (*viper.Viper, error) {
    v := viper.New()
    v.SetConfigType(fileType)
    v.SetConfigName(filename)
    v.AddConfigPath(".")
    v.AutomaticEnv()

    err := v.ReadInConfig()
    if err != nil {
        log.Printf("Unable to read config: %v", err)
        if _, ok := err.(viper.ConfigFileNotFoundError); ok {
            return nil, errors.New("config file not found")
        }
        return nil, err
    }
    return v, nil
}
Enter fullscreen mode Exit fullscreen mode
5.3 Parse Viper config

In this section, we should Unmarshal Viper config to the custom Config type declared in section 4. If this instruction was successful means the file is parsed and can be used.

func ParseConfig(v *viper.Viper) (*Config, error) {
    var cfg Config
    err := v.Unmarshal(&cfg)
    if err != nil {
        log.Printf("Unable to parse config: %v", err)
        return nil, err
    }
    return &cfg, nil
}
Enter fullscreen mode Exit fullscreen mode
5.4 Get config file from other layers

Finally, we write a function that is exposed to other layers to using config struct. This function returns a pointer of the config struct. Furthermore, some swagger port settings are handled in this function and I will explain about it in the next sections.

func GetConfig() *Config {
    cfgPath := getConfigPath(os.Getenv("APP_ENV"))
    v, err := LoadConfig(cfgPath, "yml")
    if err != nil {
        log.Fatalf("Error in load config %v", err)
    }

    cfg, err := ParseConfig(v)
    envPort := os.Getenv("PORT")
    if envPort != ""{
        cfg.Server.ExternalPort = envPort
        log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
    }else{
        cfg.Server.ExternalPort = cfg.Server.InternalPort
        log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
    }
    if err != nil {
        log.Fatalf("Error in parse config %v", err)
    }

    return cfg
}
Enter fullscreen mode Exit fullscreen mode

This is the final version of the config.go file:

package config

import (
    "errors"
    "log"
    "os"
    "time"

    "github.com/spf13/viper"
)

type Config struct {
    Server   ServerConfig
    Postgres PostgresConfig
    Redis    RedisConfig
    Password PasswordConfig
    Cors     CorsConfig
    Logger   LoggerConfig
    Otp      OtpConfig
    JWT      JWTConfig
}

type ServerConfig struct {
    InternalPort    string
    ExternalPort    string
    RunMode string
}

type LoggerConfig struct {
    FilePath string
    Encoding string
    Level    string
    Logger   string
}

type PostgresConfig struct {
    Host            string
    Port            string
    User            string
    Password        string
    DbName          string
    SSLMode         string
    MaxIdleConns    int
    MaxOpenConns    int
    ConnMaxLifetime time.Duration
}

type RedisConfig struct {
    Host               string
    Port               string
    Password           string
    Db                 string
    DialTimeout        time.Duration
    ReadTimeout        time.Duration
    WriteTimeout       time.Duration
    IdleCheckFrequency time.Duration
    PoolSize           int
    PoolTimeout        time.Duration
}

type PasswordConfig struct {
    IncludeChars     bool
    IncludeDigits    bool
    MinLength        int
    MaxLength        int
    IncludeUppercase bool
    IncludeLowercase bool
}

type CorsConfig struct {
    AllowOrigins string
}

type OtpConfig struct {
    ExpireTime time.Duration
    Digits     int
    Limiter    time.Duration
}

type JWTConfig struct {
    AccessTokenExpireDuration  time.Duration
    RefreshTokenExpireDuration time.Duration
    Secret                     string
    RefreshSecret              string
}

func GetConfig() *Config {
    cfgPath := getConfigPath(os.Getenv("APP_ENV"))
    v, err := LoadConfig(cfgPath, "yml")
    if err != nil {
        log.Fatalf("Error in load config %v", err)
    }

    cfg, err := ParseConfig(v)
    envPort := os.Getenv("PORT")
    if envPort != ""{
        cfg.Server.ExternalPort = envPort
        log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
    }else{
        cfg.Server.ExternalPort = cfg.Server.InternalPort
        log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
    }
    if err != nil {
        log.Fatalf("Error in parse config %v", err)
    }

    return cfg
}

func ParseConfig(v *viper.Viper) (*Config, error) {
    var cfg Config
    err := v.Unmarshal(&cfg)
    if err != nil {
        log.Printf("Unable to parse config: %v", err)
        return nil, err
    }
    return &cfg, nil
}
func LoadConfig(filename string, fileType string) (*viper.Viper, error) {
    v := viper.New()
    v.SetConfigType(fileType)
    v.SetConfigName(filename)
    v.AddConfigPath(".")
    v.AutomaticEnv()

    err := v.ReadInConfig()
    if err != nil {
        log.Printf("Unable to read config: %v", err)
        if _, ok := err.(viper.ConfigFileNotFoundError); ok {
            return nil, errors.New("config file not found")
        }
        return nil, err
    }
    return v, nil
}

func getConfigPath(env string) string {
    if env == "docker" {
        return "/app/config/config-docker"
    } else if env == "production" {
        return "/config/config-production"
    } else {
        return "/config/config-development"
    }
}
Enter fullscreen mode Exit fullscreen mode

In the end, I want to load the config from the config file and read the webserver port from the config instead of hard-coding.
For example, I made some changes in src/api/api.go file. First append cfg := config.GetConfig() to start of InitServer function and then load the webserver port from config r.Run(fmt.Sprintf(":%s", cfg.Server.Port)).
This is the final version of api.go file:

package api

import (
    "fmt"

    "github.com/gin-gonic/gin"
    "github.com/naeemaei/golang-clean-web-api/api/routers"
    "github.com/naeemaei/golang-clean-web-api/config" // Added
)


func InitServer(){
    cfg := config.GetConfig()  // Added
    r := gin.New()
    r.Use(gin.Logger(), gin.Recovery())

    v1 := r.Group("/api/v1/")
    {
        health := v1.Group("/health")
        routers.Health(health)
    }

    if err := r.Run(fmt.Sprintf(":%s", cfg.Server.InternalPort)); err != nil {
        panic(err)
    } // Added
}
Enter fullscreen mode Exit fullscreen mode

Now we can run the project with the go run cmd/main.go command from src directory.

Project execution result
In the next article, I will make a simple endpoint and explain how to validate the requests and how works validators in gin.

Project until here: Configuration management usingViper

gin Article's
30 articles in total
Favicon
Developing a Simple RESTful API with Gin, ginvalidator, and validatorgo
Favicon
Go's Concurrency Decoded: Goroutine Scheduling
Favicon
🚀 Building a RESTful API in Go: A Practical Guide
Favicon
A Deep Dive into Gin: Golang's Leading Framework
Favicon
Building a Blog API with Gin, FerretDB, and oapi-codegen
Favicon
How to enable hot reload in your Gin project
Favicon
Implementing an Order Processing System: Part 1 - Setting Up the Foundation
Favicon
Gin and router example
Favicon
How to Upload Images to AWS S3 with Golang
Favicon
Basic CRUD Operations Using Golang, Gin Gonic, and GORM
Favicon
Simplifying User Management with GIN and MongoDB
Favicon
Gin + Gorm Practical Guide, Implementing a Simple Q&A Community Backend Service in One Hour
Favicon
A Beginner-friendly Approach to Developing a REST API with Go, Gin and MSQL
Favicon
Cara menggunakan Cobra untuk menjalankan server Golang Gin
Favicon
A Beginner-friendly Approach to Developing a REST API with Go and Gin
Favicon
Easily build a simple and reliable ordering system in an hour using go efficiency tools
Favicon
Example 5, Automatically generate grpc gateway service project code, easy to achieve cross-service grpc calls
Favicon
Example 3, Automatically generate generic web service (gin) project code, increasing development efficiency by at least 1x
Favicon
Example 6, Build a simple golang e-commerce microservices framework step by step using tool
Favicon
Building a simple API with Golang using Gin-gonic
Favicon
Golang Web API: Project configuration management using Viper
Favicon
Golang Web API Course: Create starter project with a simple health check endpoint
Favicon
API validation in Gin: Ensuring Data Integrity in Your API
Favicon
Using the tool to complete the conversion of a community back-end single service to a microservice cluster in one day
Favicon
A secret weapon to improve the efficiency of golang development, a community backend service was developed in one day
Favicon
Play Microservices: Api-gateway service
Favicon
Set up a Stripe Checkout REST API (+ metadata) using Go and Gin Framework.
Favicon
Go: CRUD API using Gin Framework
Favicon
Example 1, Automatically generate a complete gin+gorm+redis+CRUD web service project without writing a line of Go code
Favicon
Not a Go LiveView developer yet? Try to guess what this code is doing, though.

Featured ones:

Favicon
Open
abubakersiddique761
Favicon
Open
0x3d_site
Favicon
Open
0x2e_tech
Favicon
Open
0x4c-quest
Favicon
Open
gittech