Metadata-Version: 2.4
Name: mdsi-transceiver-tools
Version: 1.0.1
Summary: SONiC transceiver EEPROM coding and diagnostic tools
Home-page: https://github.com/mdsi/sonic-transceiver-tools
Author: MDSI Engineering
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: System Administrators
Classifier: Topic :: System :: Networking
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: pyaml>=20.4.0
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: pytest-cov>=2.0; extra == "dev"
Requires-Dist: black>=21.0; extra == "dev"
Requires-Dist: flake8>=3.9; extra == "dev"
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# MDSI SONiC Transceiver Tools

A production-grade toolset for reading, parsing, and programming optical transceiver EEPROMs on SONiC network operating system platforms. Includes a Python CLI, a Node.js web dashboard, and a Switch Agent for live SSH-based switch management — all designed to run safely in the PMON container.

---

## What It Does

Network engineers often need to inspect or reprogram optical transceiver modules (DAC, AOC, AEC, optical) on SONiC switches. This tool provides:

- **Safe EEPROM reads and writes** with multi-layer validation and automatic backup
- **CMIS 4.0/5.0 and SFF-8636 protocol parsing** for transceiver identification and diagnostics
- **Profile-based coding workflows** using JSON/YAML configs — repeatable and auditable
- **A web dashboard** for switch connection management and job submission
- **A Switch Agent API** for real-time port monitoring and policy-validated optic programming via SSH

---

## Architecture

```
Browser (Web UI)
    │
    ▼  HTTP :8000
Web Server  (server.js)
    │
    ▼  HTTP :18888
Switch Agent API  (switch-agent/agent.js)
    │
    ▼  SSH
SONiC Switch  ──►  PMON Container  ──►  mdsi-transceiver CLI  ──►  Transceiver EEPROM
```

### Components

| Component | Path | Purpose |
|---|---|---|
| Web Server | `server.js` | Serves the web UI on port 8000 |
| Switch Agent | `switch-agent/agent.js` | REST API on port 18888 for switch operations |
| SSH Client | `switch-agent/ssh-client.js` | SSH2-based connection management |
| Python CLI | `src/mdsi_transceiver_tools/cli.py` | 6-command CLI for EEPROM operations |
| EEPROM Layer | `src/mdsi_transceiver_tools/eeprom_access.py` | SONiC platform API + sysfs fallback |
| Parser | `src/mdsi_transceiver_tools/cmis_parser.py` | CMIS/SFF-8636 protocol decoding |
| Coder | `src/mdsi_transceiver_tools/coder.py` | Coding workflow orchestrator with safety checks |
| Profiles | `src/mdsi_transceiver_tools/coding_profiles/` | JSON/YAML operation definitions |

---

## Getting Started

### Requirements

- **Node.js** 18+
- **Python** 3.7+
- Access to a SONiC switch (or use mock mode for testing)

### Installation

```bash
git clone https://github.com/mdsi/sonic-transceiver-tools
cd sonic-transceiver-tools

# Install Python package
pip install -e .

# Install Node dependencies
npm install
cd switch-agent && npm install && cd ..
```

### Start All Services

```bash
./start-all.sh
```

This launches both the web server (port 8000) and the Switch Agent (port 18888).

Open your browser at: **http://localhost:8000**

---

## Web Dashboard

The web UI provides four main screens:

| Screen | URL | Purpose |
|---|---|---|
| Home | `/` | Overview, downloads, quick start guide |
| Dashboard | `/dashboard.html` | Device status, profiles, operations, logs |
| Switch Connect | `/switch-connect.html` | SSH connection setup and testing |
| Ports | `/switch-ports.html` | Real-time port and module monitoring |
| Job | `/switch-job.html` | Policy-validated optic programming workflow |

### Connecting to a Switch

1. Go to **Switch Connect** in the web UI
2. Enter host, port, username, and authentication (password or SSH key)
3. Click **Test Connection** — the switch hostname and SONiC version will be displayed
4. Navigate to **Ports** to view all ports with live module data

### Running a Programming Job

1. From the Ports dashboard, select a port
2. Choose a coding profile
3. Run **Dry Run** to preview changes without writing to hardware
4. Submit with the confirmation token for the live write

See [docs/SWITCH_INTEGRATION.md](docs/SWITCH_INTEGRATION.md) for detailed workflow documentation.

---

## Python CLI

Runs inside the SONiC PMON container:

```bash
docker exec -it pmon bash
mdsi-transceiver <command> [options]
```

### Commands

#### `show` — Parsed module information

```bash
mdsi-transceiver show --port Ethernet0
mdsi-transceiver show --port Ethernet0 --format json
mdsi-transceiver show --port Ethernet0 --format compact
```

#### `info` — Quick module identification

```bash
mdsi-transceiver info --port Ethernet0
```

#### `dump` — Raw EEPROM data

```bash
mdsi-transceiver dump --port Ethernet0 --page 0 --format hexdump
mdsi-transceiver dump --port Ethernet0 --page 0 --format raw > eeprom.bin
```

#### `list-profiles` — Available coding profiles

```bash
mdsi-transceiver list-profiles
mdsi-transceiver list-profiles --verbose
```

#### `validate` — Check module/profile compatibility

```bash
mdsi-transceiver validate --port Ethernet0 --profile generic_dac_vendor_update
```

#### `code` — Program transceiver EEPROM

```bash
# Always dry-run first
mdsi-transceiver code --port Ethernet0 --profile generic_dac_vendor_update --dry-run

# Live write with interactive confirmation
mdsi-transceiver code --port Ethernet0 --profile generic_dac_vendor_update

# Non-interactive (automation/scripts)
mdsi-transceiver code --port Ethernet0 --profile generic_dac_vendor_update --yes
```

---

## Coding Profiles

Profiles define EEPROM modifications in a structured, reusable way.

### Profile Structure

```json
{
  "name": "generic_dac_vendor_update",
  "version": "1.0",
  "description": "Update vendor information for DAC cables",
  "matching": {
    "vendor": null,
    "part_number": null,
    "module_type": "dac"
  },
  "preconditions": [
    {
      "page": 0,
      "offset": 0,
      "expected": "18",
      "description": "Check QSFP-DD identifier"
    }
  ],
  "modifications": [
    {
      "page": 0,
      "offset": 129,
      "value": "4d445349202020202020202020202020",
      "description": "Set vendor name to MDSI"
    }
  ],
  "postconditions": [
    {
      "page": 0,
      "offset": 129,
      "expected": "4d445349",
      "description": "Verify vendor name was written"
    }
  ]
}
```

Add new profiles as `.json` or `.yaml` files in `src/mdsi_transceiver_tools/coding_profiles/`.

---

## Safety Features

Every EEPROM write operation goes through 8 layers of validation:

1. **Dry-Run Mode** — simulate the full operation without touching hardware
2. **Module Presence Check** — abort if module is absent or not detected
3. **Temperature Monitoring** — abort if module temperature is out of range
4. **Precondition Validation** — verify expected EEPROM state before writing
5. **Automatic Backup** — snapshot the affected pages before any modification
6. **Write Verification** — read back and compare after every write
7. **Postcondition Validation** — verify expected state after modification
8. **Audit Logging** — all operations logged with timestamps and full data

---

## Switch Agent API

The Switch Agent exposes a REST API on port 18888 for managing switch sessions and operations.

### Key Endpoints

| Method | Endpoint | Description |
|---|---|---|
| `POST` | `/switch/connect` | Establish SSH session to a SONiC switch |
| `GET` | `/switch/ports` | List all ports with module data |
| `POST` | `/switch/port/:name/admin` | Set port admin state |
| `POST` | `/switch/port/:name/power` | Set low-power mode |
| `POST` | `/switch/port/:name/reset` | Reset module |
| `GET` | `/audit-log` | Retrieve operation history |
| `POST` | `/mode` | Switch between live and mock mode (per session) |

All requests require an `X-Session-Id` header for session tracking.

See [switch-agent/README.md](switch-agent/README.md) for full API documentation.

---

## Development

### Running Tests

```bash
pip install -r requirements-dev.txt
pytest
pytest -v
pytest tests/test_parser.py
pytest --cov=mdsi_transceiver_tools
```

### Code Style

```bash
black src/ tests/
flake8 src/ tests/
mypy src/
```

### Mock Mode (No Switch Needed)

The Switch Agent supports a per-session mock mode that returns realistic test data without an SSH connection:

```bash
curl -X POST http://localhost:18888/mode \
  -H "Content-Type: application/json" \
  -H "X-Session-Id: your-session-id" \
  -d '{"mode":"mock"}'
```

---

## Project Structure

```
mdsi-transceiver-tools/
├── server.js                          # Web server (port 8000)
├── start-all.sh                       # Starts all services
├── setup.py                           # Python package definition
├── requirements.txt                   # Python runtime dependencies
├── requirements-dev.txt               # Python dev dependencies
│
├── src/mdsi_transceiver_tools/        # Python package
│   ├── cli.py                         # CLI entrypoint (6 commands)
│   ├── eeprom_access.py               # EEPROM abstraction layer
│   ├── cmis_parser.py                 # CMIS/SFF-8636 parser
│   ├── coder.py                       # Coding workflow engine
│   ├── constants.py                   # Protocol constants
│   ├── exceptions.py                  # Custom exceptions
│   ├── logging_utils.py               # Logging and audit trail
│   └── coding_profiles/               # JSON/YAML profile definitions
│
├── switch-agent/                      # Switch Agent API server
│   ├── agent.js                       # Express REST server (port 18888)
│   ├── ssh-client.js                  # SSH2 connection management
│   ├── mock-data.js                   # Mock SONiC data for testing
│   └── README.md
│
├── web/                               # Web UI
│   ├── index.html                     # Landing page
│   ├── dashboard.html                 # Main dashboard
│   ├── switch-connect.html            # Switch connection setup
│   ├── switch-ports.html              # Port monitoring
│   └── switch-job.html                # Job submission and policy validation
│
├── tests/                             # Python test suite
├── docs/                              # Extended documentation
│   ├── CLI_USAGE.md
│   ├── INTEGRATION.md
│   ├── SWITCH_INTEGRATION.md
│   └── ENGINEERING_HOWTO.md
└── scripts/                           # Build and install scripts
```

---

## Logging

All operations are logged with full context.

```bash
# Verbose console output
mdsi-transceiver show --port Ethernet0 --verbose

# Log to file
mdsi-transceiver code --port Ethernet0 --profile my_profile --log-file /tmp/transceiver.log

# Log to syslog
mdsi-transceiver code --port Ethernet0 --profile my_profile --syslog
```

Example log output:
```
2024-01-15 10:30:45 INFO  CODING START [LIVE]: port=Ethernet0, profile=generic_dac_vendor_update
2024-01-15 10:30:45 WARN  EEPROM WRITE: port=Ethernet0, page=0, offset=0x81, data=4d445349...
2024-01-15 10:30:46 INFO  CODING COMPLETE [SUCCESS]: port=Ethernet0, profile=generic_dac_vendor_update
```

---

## Troubleshooting

**Module not present**
```
Error: No module present in port Ethernet0
```
Verify the module is physically installed and detected by SONiC (`show interfaces transceiver presence`).

**Platform API unavailable**
```
WARNING: SONiC Platform API unavailable — using sysfs fallback
```
This is informational. The tool will continue using sysfs. No action needed.

**Precondition failed**
```
Error: Precondition failed: Check QSFP-DD identifier — expected 18, got 11
```
The module type does not match the profile. Run `validate` to check compatibility before coding.

**Write verification failed**
```
Error: Verification failed: wrote 4d445349, read 4d445350
```
The EEPROM write did not persist. Check for write protection or hardware issues.

---

## Documentation

| File | Contents |
|---|---|
| [QUICKSTART.md](QUICKSTART.md) | 5-minute setup guide |
| [INSTALL.md](INSTALL.md) | Full installation instructions |
| [USAGE_EXAMPLES.md](USAGE_EXAMPLES.md) | Real-world command examples |
| [docs/CLI_USAGE.md](docs/CLI_USAGE.md) | Complete CLI reference |
| [docs/SWITCH_INTEGRATION.md](docs/SWITCH_INTEGRATION.md) | Switch workflow documentation |
| [docs/INTEGRATION.md](docs/INTEGRATION.md) | SONiC platform integration details |
| [docs/ENGINEERING_HOWTO.md](docs/ENGINEERING_HOWTO.md) | Profile development and advanced topics |
| [switch-agent/README.md](switch-agent/README.md) | Switch Agent API reference |

---

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for development guidelines, coding standards, and how to submit changes.

---

## License

Apache License 2.0 — see [LICENSE](LICENSE) for details.

---

## Support

- **GitHub Issues**: Open an issue for bugs or feature requests
- **Engineering Team**: Contact MDSI Engineering for internal support
