Metadata-Version: 2.4
Name: yaml_arg_configer
Version: 0.1.5
Summary: Simplifies the process of parsing command-line arguments and configurations from YAML files
Home-page: https://github.com/eiPI1-0/yaml_arg_configer
Author: eipi10
Author-email: yorosikunano@gmail.com
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml
Dynamic: license-file

# yaml-arg-configer

A Python library for combining YAML configuration files with command-line arguments. It provides a flexible configuration management system with support for multiple config files, context variables, module imports, and function calls.

---

## Table of Contents

- [Installation](#installation)
- [Overview](#overview)
- [Quick Start](#quick-start)
- [Configuration Structure](#configuration-structure)
- [Custom YAML Tags](#custom-yaml-tags)
- [Command-Line Options](#command-line-options)
- [Troubleshooting](#troubleshooting)
- [License](#license)

---

## Installation

```bash
# Install from PyPI (when available)
pip install yaml-arg-configer

# Install from source
pip install -e .
```

---

## Overview

The `YamlArgParser` class bridges YAML configuration files and command-line arguments. It:

1. Loads default values from a YAML file
2. Merges with user-provided configurations
3. Applies command-line overrides
4. Supports dynamic value resolution via custom YAML tags

### Features

| Feature | Description |
|---------|-------------|
| **Default Configuration** | Load base settings from YAML files with `default` and `help` fields |
| **Command-Line Override** | Override settings via CLI arguments (`-c`, `-d`, etc.) |
| **Multiple Configurations** | Combine multiple YAML config files in priority order |
| **Context Resolution** | Reference context variables using `!ctx` tag |
| **Module Imports** | Import attributes from Python modules using `!ipt` tag |
| **Function Calls** | Execute Python functions using `!call` tag |
| **Path Operations** | Use `!join` and `!chain2path` for path manipulation |
| **Config Saving** | Export current configuration back to YAML |

---

## Quick Start

### Basic Example

```python
from yaml_arg_configer import YamlArgParser

# Initialize parser with default YAML file
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse command-line arguments
cmd_args = yaml_parser.parse_cmd_args()

# Parse configurations from YAML files
final_args = yaml_parser.parse_args(cmd_args)

# Access results
print(yaml_parser.get_args())           # argparse.Namespace
print(yaml_parser.get_args_dict())       # dict
print(yaml_parser['arg_name'])           # dict-like access
print(yaml_parser.get('arg_name'))       # dict-like access with default value, return None by default if arg not exist
yaml_parser.save_config('output.yaml')   # Save to YAML
```

### Programmatic Usage

```python
from yaml_arg_configer import YamlArgParser

# Initialize parser
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse defaults
yaml_parser.parse_default('defaults.yaml')

# Update from additional YAML files
yaml_parser.update_from_yaml('config1.yaml', strict=True)
yaml_parser.update_from_yaml('config2.yaml', strict=True)

# Get final arguments as dict
final_args = yaml_parser.get_args_dict()

# Or as argparse.Namespace
final_args = yaml_parser.get_args()
```

### Saving Configuration

```python
# Save current configuration to YAML
yaml_parser.save_config('output_config.yaml')
```

### Parsing Configurations from Directory

The `parse_cfgs()` method is convenient for loading configurations from a directory:

```python
# Initialize parser
yaml_parser = YamlArgParser(default_yaml='defaults.yaml')

# Parse configurations from files in a directory
# Loads: config1.yaml, config2.yaml, etc.
final_args = yaml_parser.parse_cfgs(
    cfgs=['config1', 'config2'],  # Config file names (without .yaml)
    cfg_dir='configs',             # Directory containing config files
    strict=True                    # Enforce validation
)

# Access results
print(final_args)  # Returns dict with parsed arguments
```

**Key Features:**
- Automatically constructs file paths: `{cfg_dir}/{name}.yaml`
- Loads context from `ctx.yaml` if specified
- Parses defaults before applying config files
- Returns merged configuration as dictionary

---

## Configuration Structure

### Default Configuration (`defaults.yaml`)

```yaml
arg1:
  default: ~                    # ~ means None
  help: "Description for arg1"
arg2:
  default: 42
  help: "Number of iterations"
arg3: True                      # Simple format: value directly
arg4: "string value"            # Simple format: string value
```

### User Configuration (`config1.yaml`)

```yaml
arg1: value1                    # Override default value
arg2: 100                       # Override another value
```

### Context Management

Context YAML files support dynamic context management:

```yaml
# ctx.yaml
import: !ipt
  torch.nn:
    relu: ReLU
    gelu: GELU
```

The `!ipt` tag can import multiple modules and attributes, which are then available for `!ctx` references.

---

## Advanced: Custom YAML Tags

The library extends PyYAML with custom tags for dynamic value resolution.

### `!ctx` - Context Variables

Reference variables defined in a context YAML file:

**Context file (`ctx.yaml`):**
```yaml
import: !ipt
  torch.nn:
    relu: ReLU
    gelu: GELU
```

**Usage in config:**
```yaml
activation: !ctx relu           # Resolves to "ReLU"
layer: !ctx gelu                # Resolves to "GELU"
```

### `!ipt` - Import Attributes

Import attributes from Python modules:

```yaml
import: !ipt
  datetime: [now]               # Resolves to datetime.datetime.now()
  os: [path, join]              # Resolves to os.path.join
  math: [sqrt, pi]              # Resolves to math.sqrt, math.pi
```

### `!call` - Function Calls

Execute Python functions and use the result:

```yaml
output_path: !call
  call: os.path.join
  part1: tmp
  part2: output
```

Resolves to: `tmp/output`

### `!join` - Join Lists

Flatten and join list elements:

```yaml
paths:
  - /usr/local
  - bin
```

Resolves to: `['/usr/local', 'bin']`

### `!chain2path` - Path Chain

Chain multiple path components:

```yaml
output: !chain2path
  - tmp
  - output
  - data
```

Resolves to: `tmp/output/data`

---

## Command-Line Options

| Flag | Description | Example |
|------|-------------|---------|
| `-d, --cfg_dir` | Directory containing YAML config files | `-d configs` |
| `-c, --cfg` | Names of configuration files to load | `-c config1 config2` |
| `-dc, --default_yaml` | Path to default YAML file | `-dc defaults.yaml` |
| `-ctx, --ctx_yaml` | Path to context YAML file | `-ctx ctx.yaml` |
| `--strict` | Enforce user config keys exist in defaults | `--strict` |
| `--doc <arg>` | Get help for specific argument(s) | `--doc arg1 arg2` |
| `--docs` | Print help for all arguments | `--docs` |

### Example Commands

```bash
# Load defaults and config files
python my_script.py -dc defaults.yaml -d configs -c config1 config2

# With context resolution
python my_script.py -dc defaults.yaml -ctx ctx.yaml -d configs -c config1

# Get help documentation
python my_script.py --docs
python my_script.py --doc arg1 arg2

# Strict mode (enforce config validation)
python my_script.py -dc defaults.yaml --strict -d configs -c config1
```

---

## Troubleshooting

### Best Practices

1. **Start with defaults**: Define comprehensive defaults in `defaults.yaml`
2. **Use context files**: Centralize reusable values in `ctx.yaml`
3. **Document with help**: Always provide `help` text for arguments
4. **Validate with strict**: Use `--strict` to catch configuration errors
5. **Chain configs**: Load multiple configs for modular configuration

### Common Issues

**Missing keys in strict mode**
- Symptom: `AssertionError: Argument 'X' not defined in the default configuration.`
- Solution: Ensure all user config keys appear in `defaults.yaml` with proper structure (even if using minimal entries)

**Context variables not resolving**
- Symptom: `!ctx xxx` stays as literal string instead of resolving
- Check that `ctx.yaml` exists and is loaded via `-ctx` flag
- Verify import paths are correct in context yaml (`import: !ipt`)
- Variable names match between import and reference

**Config merging order wrong**
- Symptom: Config file values overwritten by defaults instead of overriding them
- Explanation: Priority order is: CLI args > user configs > defaults. Ensure config files load after defaults

---

## License

MIT
