outbound v0.5.2 [base]
Главный шлюз внешних подключений
| version | date | commit | файлов |
|---|---|---|---|
| 0.5.2 | 2026-09-17 | 46803aa3a927 | 17 |
README
# outbound
Шлюз внешних подключений: точки, потребители, привязки, health, журнал.
Тип: Модуль. Категория: `base`. Зависимости: pip-пакеты `urllib3`, `paho-mqtt`, `pysmb`, `pywinrm` (по видам точек); системные `openssh-client`, `sshpass`, `iputils-ping`, `nfs-common` (ssh/sftp, ping, NFS).
## Описание
Правило простое: ни один модуль не ходит наружу сам — все ходят через шлюз.
Шлюз оперирует тремя сущностями. **Точка** — это «куда подключаться»: вид, адрес и опции. **Потребитель** — модуль, который заявляет о своей потребности сам при старте. **Привязка** — связь «кто к какой точке». Вызов без привязки отклоняется с подсказкой, что привязать.
Секреты (пароли и ключи) живут только в `.env` модуля и подтягиваются по ссылке точки. В самих файлах точек секретов нет. Отключение проверки TLS действует только для локальных адресов.
Точка умеет ходить через промежуточный хост и через прокси, есть сухой прогон без выполнения. Подключение проверяется пробным вызовом ещё до сохранения. Здоровье всех точек проверяется фоном по расписанию, а каждый вызов пишется в кольцевой журнал.
## Возможности
- `endpoint_list {}` → все точки подключения (без секретов)
- `endpoint_add {name, kind, address, opts?, auth_ref?, via?, proxy?, health?, note?}` → добавить точку (с пробным подключением до сохранения)
- `endpoint_update {name, patch: {...}}` → частичное обновление точки
- `endpoint_delete {name}` → удалить точку (привязки к ней сбрасываются)
- `endpoint_test {name}` → ручная проверка health точки
- `consumer_list {}` → клиенты-потребители и их привязки
- `consumer_ensure {module, need, desc}` → заявить потребность (первый вызов — статус pending)
- `bind {module, endpoint|null}` → привязать клиента к точке (null — отвязать)
- `unbind {module}` → снять привязку (возврат в pending)
- `call {consumer, op, params?}` → единый вызов наружу от имени модуля (требует привязки)
- `health {}` → сводка health всех точек
- `log {limit?}` → кольцевой журнал вызовов (последние N)
- `kinds {}` → поддерживаемые kind точек
- `endpoint_base {consumer, field=base_url}` → адрес привязанной точки для потребителя
## Права
- `outbound_manage` — Точки подключения и привязки клиентов (кабинет outbound)
- `outbound_view` — Просмотр точек, health и журнала outbound
## Web
- `GET /api/outbound/endpoints` — точки подключения
- `POST /api/outbound/endpoint` — добавить/обновить точку
- `POST /api/outbound/endpoint_test` — проверка точки
- `GET /api/outbound/consumers` — потребители и привязки
- `POST /api/outbound/bind` — привязка клиента к точке
- `GET /api/outbound/health` — здоровье точек
- `GET /api/outbound/log` — журнал вызовов
- `POST /api/outbound/call` — отладочный вызов
## Конфигурация (`config.schema.json`)
- Опциональные: `HEALTH_INTERVAL` = `60`, `LOG_LIMIT` = `200`, `DEFAULT_TIMEOUT` = `15`
Манифест
{
"name": "outbound",
"version": "0.5.2",
"category": "base",
"description": "Шлюз внешних подключений: точки (ssh/http/tcp/udp/icmp/dns/wol/mqtt/ws/sftp/smb/ftp/nfs/winrm/docker/proxmox/telnet), клиенты-потребители, привязки, health, журнал. Режимы: via (jump-host), proxy, dry-run. Фронтенд — web/view.html+view.js (модульная вьюха).",
"requires": [],
"rights": [],
"api_methods": [],
"web_routes": [],
"commit": "46803aa3a927a3a188c2249f494e5ea5ecf92dc6",
"updated": "2026-09-17 12:54:21 +0300",
"integrity": "sha256:fbe86d7a9cc6357e03f0cf0c8be3f91e19701ec623b3abe517b62d6f1a4da88d",
"files": "[17 файлов — см. вкладки ниже]",
"note": "Главный шлюз внешних подключений",
"wdesc": "Главный шлюз внешних подключений",
"wbody": "",
"_extra": {
"api": "",
"events_emitted": "[]",
"events_subscribed": "[]",
"audit_allow": "",
"shared_routes": "",
"config_schema": "config.schema.json",
"env_example": ".env.example"
}
} Файлы и исходники
Дерево файлов
- · корень
- 4.2 КБ
- 0.1 КБ
- 4.0 КБ
- 17.1 КБ
- 1.0 КБ
- 1.8 КБ
- 4.7 КБ
- drivers/
- 3.1 КБ
- 2.4 КБ
- 13.5 КБ
- 12.4 КБ
- 10.7 КБ
- 8.5 КБ
- 7.8 КБ
- 3.6 КБ
- web/
- 2.7 КБ
- 9.2 КБ
Предпросмотр
Выберите файл в дереве выше — код откроется здесь.
Все исходники (.py) одним списком
drivers/__init__.py 3.1 КБ
"""outbound.drivers — реестр: kind -> драйвер, op -> драйвер.
Сквозные режимы применяются здесь, а не в драйверах:
via: имя другой точки kind=ssh — ssh идёт через ProxyJump (-J);
остальным kind via пока запрещён (явная ошибка, не молча).
proxy: только kind=http (opts.proxy); остальным — явная ошибка.
dryrun игнорирует via/proxy.
"""
import logging
from . import base as _b
from . import file as _file
from . import http as _http
from . import mgmt as _mgmt
from . import msg as _msg
from . import net as _net
from . import ssh as _ssh
logger = logging.getLogger(__name__)
_DRIVERS = (_net, _http, _ssh, _msg, _file, _mgmt)
KIND2DRV = {}
OP2DRV = {}
for _d in _DRIVERS:
for _k in getattr(_d, "KINDS", ()):
KIND2DRV[_k] = _d
for _op in getattr(_d, "OPS", {}):
OP2DRV[_op] = _d
def kinds():
return sorted(KIND2DRV)
def ops():
return sorted(OP2DRV)
def dispatch(ep, secret, op, params, via_jump=None):
"""Выполнить op на точке. Возвращает dict (всегда, без исключений)."""
kind = (ep.get("kind") or "").strip()
drv = KIND2DRV.get(kind)
if drv is None:
return _b.err("неизвестный kind точки: %s" % kind)
if op not in getattr(drv, "OPS", {}):
return _b.err("точка %s (%s) не умеет op=%s" % (
ep.get("name"), kind, op))
if ep.get("proxy") and kind != "http":
return _b.err("proxy поддерживается только kind=http")
if via_jump and kind != "ssh":
return _b.err("via (jump-host) поддерживается только kind=ssh")
fn = drv.OPS[op]
try:
if via_jump and kind == "ssh":
return fn(ep, secret, params or {}, jump=via_jump)
return fn(ep, secret, params or {})
except TypeError:
# драйвер без jump-kwarg (на будущее) — обычный вызов
try:
return fn(ep, secret, params or {})
except Exception as e: # noqa: BLE001
logger.exception("outbound.dispatch %s", op)
return _b.err("внутренняя ошибка драйвера: %s" % e)
except Exception as e: # noqa: BLE001
logger.exception("outbound.dispatch %s", op)
return _b.err("внутренняя ошибка драйвера: %s" % e)
def check_health(ep, secret, via_jump=None):
drv = KIND2DRV.get((ep.get("kind") or "").strip())
if drv is None or not hasattr(drv, "health"):
return _b.err("нет health для kind=%s" % ep.get("kind"))
try:
if via_jump:
return drv.health(ep, secret, jump=via_jump)
return drv.health(ep, secret)
except TypeError:
try:
return drv.health(ep, secret)
except Exception as e: # noqa: BLE001
return _b.err("health: %s" % e)
except Exception as e: # noqa: BLE001
return _b.err("health: %s" % e)
drivers/base.py 2.4 КБ
"""outbound.drivers.base — общие помощники драйверов.
Драйвер = модуль с:
KINDS = ("kind1", ...) какие kind обслуживает
OPS = {"op": func} операции; func(ep, secret, params) -> dict
health(ep, secret) проверка точки -> {"ok": bool, "ms": int, ...}
Контракт результата операции: {"ok": True, ...} либо {"ok": False,
"error": "..."}. Секреты в результаты/ошибки не попадают.
"""
import importlib
import logging
import subprocess
import time
logger = logging.getLogger(__name__)
def now_ms():
return int(time.time() * 1000)
def err(msg, **extra):
out = {"ok": False, "error": str(msg)[:500]}
out.update(extra)
return out
def ok(**extra):
out = {"ok": True}
out.update(extra)
return out
def opt(ep, key, default=None):
try:
return (ep.get("opts") or {}).get(key, default)
except Exception: # noqa: BLE001
return default
def addr(ep, key, default=""):
try:
return (ep.get("address") or {}).get(key, default)
except Exception: # noqa: BLE001
return default
def timeout(ep, default=15):
try:
return int(opt(ep, "timeout", default) or default)
except (TypeError, ValueError):
return default
def lazy(modname):
"""Импорт опциональной зависимости: (модуль|None)."""
try:
return importlib.import_module(modname)
except Exception: # noqa: BLE001
return None
def missing(pkg, kind):
return err("драйвер %s: нет пакета %s (requirements.txt модуля "
"outbound)" % (kind, pkg))
def run(cmd, timeout_s=15, env=None, input_text=None):
"""subprocess без shell. Возвращает (rc, stdout, stderr)."""
try:
r = subprocess.run(
cmd, capture_output=True, text=True, errors="replace",
timeout=timeout_s, env=env, input=input_text)
return r.returncode, (r.stdout or "")[:8000], (r.stderr or "")[:2000]
except subprocess.TimeoutExpired:
return None, "", "timeout %ss" % timeout_s
except FileNotFoundError:
return None, "", "нет бинаря: %s" % (cmd[0] if cmd else "?")
except Exception as e: # noqa: BLE001
return None, "", "%s: %s" % (type(e).__name__, e)
drivers/file.py 13.5 КБ
"""outbound.drivers.file — sftp/smb/ftp/nfs.
sftp: бинарь OpenSSH (batchmode), без pip. smb: pysmb (опционально).
ftp: ftplib (stdlib). nfs: точка монтирования на хосте бота (nfs-common),
драйвер делает файловые операции по mount_path + проверяет, что это mount.
"""
import os as _os
from . import base as b
KINDS = ("sftp", "smb", "ftp", "nfs")
def _local_safe(local):
"""Проверка локального пути для get/put: без выхода за пределы.
Абсолютные пути разрешены (бэкапы работают с абсолютными путями
под BACKUP_DIR), но запрещены: пусто, NUL-байт, ~ (дом. каталог),
..-сегменты в любом стиле разделителей. Возвращает очищенный путь
или "" (плохой путь).
"""
p = (local or "").strip()
if not p or "\x00" in p:
return ""
if p.startswith("~"):
return ""
segs = p.replace("\\", "/").split("/")
if any(s == ".." for s in segs):
return ""
return p
def _remote_nl_ok(*vals):
"""Нет CR/LF в remote-путях (инъекция команд в ftp/sftp-batch)."""
for v in vals:
if v and ("\r" in v or "\n" in v):
return False
return True
def _ssh_target(ep):
user = (b.addr(ep, "user") or "").strip()
host = (b.addr(ep, "host") or "").strip()
try:
port = int(b.addr(ep, "port") or 22)
except (TypeError, ValueError):
port = 22
if not host:
return None
return ("%s@%s" % (user, host) if user else host), port
# --- sftp (бинарь) ---
def _sftp_batch(ep, secret, batch):
tgt = _ssh_target(ep)
if not tgt:
return None, b.err("sftp: нужен address.host")
target, port = tgt
key = (b.addr(ep, "key") or "").strip()
cmd = ["sftp", "-P", str(port), "-o",
"StrictHostKeyChecking=%s" % b.opt(ep, "strict_host_key",
"accept-new"),
"-o", "ConnectTimeout=%s" % min(b.timeout(ep), 30),
"-b", "-"]
if key:
cmd += ["-i", key]
cmd.append(target)
env = None
if secret and not key:
cmd = ["sshpass", "-e"] + cmd
import os as _o
env = dict(_o.environ)
env["SSHPASS"] = secret
rc, out, err = b.run(cmd, timeout_s=b.timeout(ep) + 30, env=env,
input_text=batch)
if rc is None:
return None, b.err("sftp: %s" % err)
if rc != 0:
return None, b.err("sftp rc=%s: %s" % (rc, (err or out)[:300]))
return out, None
def sftp_list(ep, secret, params):
path = ((params or {}).get("path") or ".").strip()
out, err = _sftp_batch(ep, secret, "ls -l %s\n" % _q(path))
if err:
return err
return b.ok(path=path, listing=out.strip()[:4000])
def sftp_get(ep, secret, params):
remote = ((params or {}).get("remote") or "").strip()
local = _local_safe((params or {}).get("local"))
if not remote or not local:
return b.err("sftp.get: нужны params.remote и params.local "
"(local — без .. и ~)")
if not _remote_nl_ok(remote, local):
return b.err("sftp.get: переводы строк в путях запрещены")
out, err = _sftp_batch(ep, secret, "get %s %s\n" % (_q(remote),
_q(local)))
if err:
return err
return b.ok(remote=remote, local=local)
def sftp_put(ep, secret, params):
local = _local_safe((params or {}).get("local"))
remote = ((params or {}).get("remote") or "").strip()
if not local or not remote or not _os.path.isfile(local):
return b.err("sftp.put: нужен существующий params.local и "
"params.remote")
if not _remote_nl_ok(remote, local):
return b.err("sftp.put: переводы строк в путях запрещены")
out, err = _sftp_batch(ep, secret, "put %s %s\n" % (_q(local),
_q(remote)))
if err:
return err
return b.ok(local=local, remote=remote)
def _q(s):
# перевод строки разорвал бы sftp-batch на две команды — режем сразу
s = str(s or "").replace("\r", "").replace("\n", "")
return "'%s'" % s.replace("'", "'\\''")
# --- smb (pysmb) ---
def _smb_conn(ep, secret):
smb = b.lazy("smb.SMBConnection")
if not smb:
return None, b.missing("pysmb", "smb")
host = (b.addr(ep, "host") or "").strip()
user = (b.addr(ep, "user") or "").strip()
share = (b.addr(ep, "share") or "").strip()
if not host or not user or not share:
return None, b.err("smb: нужны address.host/user/share")
try:
conn = smb.SMBConnection(user, secret or "", "bot", host,
use_ntlm_v2=True)
if not conn.connect(host, int(b.addr(ep, "port") or 445),
timeout=b.timeout(ep)):
return None, b.err("smb: %s не отвечает" % host)
return (conn, share), None
except Exception as e: # noqa: BLE001
return None, b.err("smb.connect: %s" % e)
def smb_list(ep, secret, params):
path = ((params or {}).get("path") or "/").strip() or "/"
conn, err = _smb_conn(ep, secret)
if err:
return err
conn_obj, share = conn
try:
items = conn_obj.listPath(share, path, timeout=b.timeout(ep))
names = [i.filename for i in items
if i.filename not in (".", "..")]
return b.ok(path=path, names=names[:200])
except Exception as e: # noqa: BLE001
return b.err("smb.list: %s" % e)
finally:
try:
conn_obj.close()
except Exception: # noqa: BLE001
pass
def smb_get(ep, secret, params):
remote = ((params or {}).get("remote") or "").strip()
local = _local_safe((params or {}).get("local"))
if not remote or not local:
return b.err("smb.get: нужны params.remote и params.local "
"(local — без .. и ~)")
conn, err = _smb_conn(ep, secret)
if err:
return err
conn_obj, share = conn
try:
with open(local, "wb") as f:
conn_obj.retrieveFile(share, remote, f, timeout=b.timeout(ep))
return b.ok(remote=remote, local=local)
except Exception as e: # noqa: BLE001
return b.err("smb.get: %s" % e)
finally:
try:
conn_obj.close()
except Exception: # noqa: BLE001
pass
def smb_put(ep, secret, params):
remote = ((params or {}).get("remote") or "").strip()
local = _local_safe((params or {}).get("local"))
if not remote or not local:
return b.err("smb.put: нужны params.remote и params.local "
"(local — без .. и ~)")
if ".." in remote.split("/"):
return b.err("smb.put: remote вне шары запрещён")
conn, err = _smb_conn(ep, secret)
if err:
return err
conn_obj, share = conn
try:
with open(local, "rb") as f:
conn_obj.storeFile(share, remote, f, timeout=b.timeout(ep))
return b.ok(remote=remote, local=local)
except Exception as e: # noqa: BLE001
return b.err("smb.put: %s" % e)
finally:
try:
conn_obj.close()
except Exception: # noqa: BLE001
pass
def smb_mkdir(ep, secret, params):
remote = ((params or {}).get("remote") or "").strip().strip("/")
if not remote:
return b.err("smb.mkdir: нужен params.remote")
if ".." in remote.split("/"):
return b.err("smb.mkdir: вне шары запрещён")
conn, err = _smb_conn(ep, secret)
if err:
return err
conn_obj, share = conn
try:
try:
conn_obj.createDirectory(share, remote, timeout=b.timeout(ep))
except Exception:
pass # уже есть — не ошибка
return b.ok(remote=remote)
except Exception as e: # noqa: BLE001
return b.err("smb.mkdir: %s" % e)
finally:
try:
conn_obj.close()
except Exception: # noqa: BLE001
pass
def smb_delete(ep, secret, params):
remote = ((params or {}).get("remote") or "").strip()
if not remote:
return b.err("smb.delete: нужен params.remote")
if ".." in remote.split("/"):
return b.err("smb.delete: remote вне шары запрещён")
conn, err = _smb_conn(ep, secret)
if err:
return err
conn_obj, share = conn
try:
conn_obj.deleteFiles(share, remote, timeout=b.timeout(ep))
return b.ok(remote=remote)
except Exception as e: # noqa: BLE001
return b.err("smb.delete: %s" % e)
finally:
try:
conn_obj.close()
except Exception: # noqa: BLE001
pass
# --- ftp (stdlib) ---
def _ftp_conn(ep, secret):
from ftplib import FTP
host = (b.addr(ep, "host") or "").strip()
user = (b.addr(ep, "user") or "anonymous").strip()
try:
port = int(b.addr(ep, "port") or 21)
except (TypeError, ValueError):
port = 21
if not host:
return None, b.err("ftp: нужен address.host")
try:
ftp = FTP()
ftp.connect(host, port, timeout=b.timeout(ep))
ftp.login(user, secret or "")
return ftp, None
except Exception as e: # noqa: BLE001
return None, b.err("ftp.connect: %s" % e)
def ftp_list(ep, secret, params):
path = ((params or {}).get("path") or "").strip()
ftp, err = _ftp_conn(ep, secret)
if err:
return err
try:
names = ftp.nlst(path or None)
return b.ok(path=path or "/", names=names[:200])
except Exception as e: # noqa: BLE001
return b.err("ftp.list: %s" % e)
finally:
try:
ftp.quit()
except Exception: # noqa: BLE001
pass
def ftp_get(ep, secret, params):
remote = ((params or {}).get("remote") or "").strip()
local = _local_safe((params or {}).get("local"))
if not remote or not local:
return b.err("ftp.get: нужны params.remote и params.local "
"(local — без .. и ~)")
if not _remote_nl_ok(remote):
# ftplib склеивает "RETR " + remote: CR/LF = инъекция команд
return b.err("ftp.get: переводы строк в remote запрещены")
ftp, err = _ftp_conn(ep, secret)
if err:
return err
try:
with open(local, "wb") as f:
ftp.retrbinary("RETR " + remote, f.write)
return b.ok(remote=remote, local=local)
except Exception as e: # noqa: BLE001
return b.err("ftp.get: %s" % e)
finally:
try:
ftp.quit()
except Exception: # noqa: BLE001
pass
# --- nfs (точка монтирования на хосте бота) ---
def _nfs_root(ep):
root = (b.addr(ep, "mount_path") or "").strip()
if not root or not _os.path.isdir(root):
return None, b.err("nfs: address.mount_path не смонтирован: %s"
% root)
if not _os.path.ismount(root):
return None, b.err("nfs: %s не mount (stale?)" % root)
return root, None
def _nfs_safe(root, rel):
rel = (rel or "").strip().lstrip("/")
full = _os.path.realpath(_os.path.join(root, rel))
if full != root and not full.startswith(root + _os.sep):
return None
return full
def nfs_list(ep, secret, params):
root, err = _nfs_root(ep)
if err:
return err
full = _nfs_safe(root, (params or {}).get("path") or "")
if not full or not _os.path.isdir(full):
return b.err("nfs.list: плохой путь")
try:
return b.ok(path=full, names=sorted(_os.listdir(full))[:200])
except Exception as e: # noqa: BLE001
return b.err("nfs.list: %s" % e)
def nfs_read(ep, secret, params):
root, err = _nfs_root(ep)
if err:
return err
full = _nfs_safe(root, (params or {}).get("path") or "")
if not full or not _os.path.isfile(full):
return b.err("nfs.read: плохой путь")
try:
if _os.path.getsize(full) > 512 * 1024:
return b.err("nfs.read: файл >512K (забирай кусками позже)")
with open(full, "r", encoding="utf-8", errors="replace") as f:
return b.ok(path=full, text=f.read()[:8000])
except Exception as e: # noqa: BLE001
return b.err("nfs.read: %s" % e)
OPS = {
"sftp.list": sftp_list,
"sftp.get": sftp_get,
"sftp.put": sftp_put,
"smb.list": smb_list,
"smb.get": smb_get,
"smb.put": smb_put,
"smb.mkdir": smb_mkdir,
"smb.delete": smb_delete,
"ftp.list": ftp_list,
"ftp.get": ftp_get,
"nfs.list": nfs_list,
"nfs.read": nfs_read,
}
def health(ep, secret):
kind = (ep.get("kind") or "").strip()
if kind == "sftp":
out, err = _sftp_batch(ep, secret, "pwd\n")
return b.ok() if err is None else err
if kind == "smb":
r = smb_list(ep, secret, {"path": (b.addr(ep, "health_path")
or "/")})
return b.ok() if r.get("ok") else r
if kind == "ftp":
r = ftp_list(ep, secret, {})
return b.ok() if r.get("ok") else r
if kind == "nfs":
root, err = _nfs_root(ep)
return b.ok(mount=root) if err is None else err
return b.err("file: нет health для kind=%s" % kind)
drivers/http.py 12.4 КБ
"""outbound.drivers.http — http/https через urllib3 (зависимость ядра де-факто).
Операции: http.request (JSON/текст), http.stream (буфер с лимитом — для
будущего SSE потребители пока читают собранное тело + чанки-мету).
Прокси: opts.proxy (http://...) либо SOCKS при наличии PySocks.
"""
import json as _json
from . import base as b
KINDS = ("http",)
def _loopback_host(host):
"""Строго loopback: localhost/::1/127.x.x.x (без префикс-матчей вида 127.evil.com)."""
h = (host or "").strip("[]").lower()
if h in ("localhost", "::1"):
return True
parts = h.split(".")
return (len(parts) == 4 and parts[0] == "127"
and all(p.isdigit() and 0 <= int(p) <= 255 for p in parts[1:]))
def _resolve_verify(ep, params, url):
"""verify для запроса: дефолт из биндинга; params verify:false —
только для loopback (самоподписанный серт панели), иначе игнор."""
verify = (params or {}).get("verify")
if verify is None:
return b.opt(ep, "verify", True)
if verify is False:
try:
host = (url or "").split("://", 1)[1].split("/", 1)[0]
host = host.split("@")[-1].split(":")[0]
except Exception: # noqa: BLE001
host = ""
if not _loopback_host(host):
return True
return False
return True
def _manager(ep, verify=True):
u3 = b.lazy("urllib3")
if not u3:
return None, b.missing("urllib3", "http")
proxy = (b.opt(ep, "proxy") or "").strip()
if proxy:
pm = u3.ProxyManager(proxy, cert_reqs="CERT_REQUIRED" if verify
else "CERT_NONE")
else:
pm = u3.PoolManager(cert_reqs="CERT_REQUIRED" if verify
else "CERT_NONE")
return pm, None
def _headers(ep, secret, params):
h = dict((b.opt(ep, "headers") or {}))
h.update((params or {}).get("headers") or {})
auth = (b.opt(ep, "auth") or "").strip().lower()
if auth == "bearer" and secret:
h["Authorization"] = "Bearer " + secret
elif auth == "basic" and secret:
import base64 as _b64
h["Authorization"] = "Basic " + _b64.b64encode(
secret.encode()).decode()
elif auth == "token" and secret:
h["Authorization"] = "token " + secret
return h
def http_request(ep, secret, params):
params = params or {}
base = (b.addr(ep, "base_url") or "").rstrip("/")
path = (params.get("path") or "/").strip() or "/"
if not path.startswith("/"):
path = "/" + path
url = (params.get("url") or (base + path)).strip()
if not url.startswith(("http://", "https://")):
return b.err("http.request: bad url")
pm, err = _manager(ep, _resolve_verify(ep, params, url))
if err:
return err
try:
timeout = int(params.get("timeout") or 0)
except (TypeError, ValueError):
timeout = 0
if timeout:
timeout = max(2, min(timeout, 300))
else:
timeout = b.timeout(ep)
method = (params.get("method") or "GET").upper()
if method not in ("GET", "POST", "PUT", "PATCH", "DELETE", "HEAD"):
return b.err("http.request: метод %s запрещён" % method)
body = params.get("json")
raw = params.get("body")
multi = params.get("multipart")
data = None
ctype = None
if multi is not None:
# {"поле": "текст" | {"filename","data_b64","content_type"}}
# (base64 — JSON-безопасно, работает и поверх HTTP API).
import base64 as _b64
from urllib3.filepost import encode_multipart_formdata as _enc
fields = {}
for k, v in (multi if isinstance(multi, dict) else {}).items():
if isinstance(v, dict) and "data_b64" in v:
try:
blob = _b64.b64decode(v.get("data_b64") or "")
except Exception:
return b.err("http.request: битый base64 в поле %s" % k)
fields[k] = (v.get("filename") or k,
blob, v.get("content_type")
or "application/octet-stream")
else:
fields[k] = str(v)
try:
data, ctype = _enc(fields)
except Exception as e: # noqa: BLE001
return b.err("http.request: multipart: %s" % e)
elif body is not None:
data = _json.dumps(body, ensure_ascii=False).encode()
ctype = "application/json"
elif raw is not None:
data = raw.encode() if isinstance(raw, str) else bytes(raw)
headers = _headers(ep, secret, params)
if ctype:
headers.setdefault("Content-Type", ctype)
t0 = b.now_ms()
try:
r = pm.request(method, url, body=data, headers=headers,
timeout=timeout, retries=0)
text = (r.data or b"")[:200000].decode("utf-8", "replace")
out = b.ok(status=r.status, ms=b.now_ms() - t0)
if "application/json" in (r.headers.get("Content-Type") or ""):
try:
out["json"] = _json.loads(text)
except Exception: # noqa: BLE001
out["text"] = text[:8000]
else:
out["text"] = text[:8000]
return out
except Exception as e: # noqa: BLE001
return b.err("http %s %s: %s" % (method, url.split("?")[0], e),
ms=b.now_ms() - t0)
def http_stream(ep, secret, params):
"""Чтение тела чанками с лимитом (задел под SSE-миграцию opencode).
Возвращает собранный текст + число чанков; настоящий построчный SSE
пока остаётся в модуле opencode.
"""
params = params or {}
try:
limit = int(params.get("limit_kb", 512))
except (TypeError, ValueError):
limit = 512
limit = max(16, min(limit, 4096)) * 1024
base = (b.addr(ep, "base_url") or "").rstrip("/")
path = (params.get("path") or "/").strip() or "/"
url = (params.get("url") or (base + path)).strip()
pm, err = _manager(ep, _resolve_verify(ep, params, url))
if err:
return err
t0 = b.now_ms()
try:
r = pm.request("GET", url, headers=_headers(ep, secret, params),
timeout=b.timeout(ep), retries=0, preload_content=False)
chunks = 0
buf = bytearray()
for chunk in r.stream(65536):
chunks += 1
buf += chunk
if len(buf) >= limit:
break
r.release_conn()
text = bytes(buf).decode("utf-8", "replace")
return b.ok(status=r.status, chunks=chunks,
truncated=len(buf) >= limit,
text=text[:8000], ms=b.now_ms() - t0)
except Exception as e: # noqa: BLE001
return b.err("http.stream %s: %s" % (url.split("?")[0], e),
ms=b.now_ms() - t0)
OPS = {
"http.request": http_request,
"http.stream": http_stream,
"http.sse": None, # подставляется ниже (генератор)
}
def sse_lines(ep, secret, params):
"""SSE/event-stream построчно: генератор data:-пейлоадов (текст).
Фрейминг — здесь, парсинг JSON — у потребителя. params: path|url,
method, json/body, headers, timeout (дедлайн всего чтения, до 900с),
max_lines, max_bytes. Закрытие генератора (close/break) рвёт соединение
(finally + release_conn). Ошибка HTTP — исключение на первом next().
"""
params = params or {}
base = (b.addr(ep, "base_url") or "").rstrip("/")
path = (params.get("path") or "/").strip() or "/"
if not path.startswith("/"):
path = "/" + path
_sse_url = (params.get("url") or (base + path)).strip()
pm, err = _manager(ep, _resolve_verify(ep, params, _sse_url))
if err:
def _e():
raise RuntimeError(err.get("error"))
yield
return _e()
try:
timeout = int(params.get("timeout") or 0)
except (TypeError, ValueError):
timeout = 0
timeout = max(5, min(timeout or 120, 900))
try:
max_lines = int(params.get("max_lines") or 50000)
except (TypeError, ValueError):
max_lines = 50000
max_lines = max(1, min(max_lines, 200000))
try:
max_bytes = int(params.get("max_bytes") or 16777216)
except (TypeError, ValueError):
max_bytes = 16777216
max_bytes = max(65536, min(max_bytes, 67108864))
url = _sse_url
if not url.startswith(("http://", "https://")):
def _e2():
raise RuntimeError("http.sse: bad url")
yield
return _e2()
method = (params.get("method") or "GET").upper()
if method not in ("GET", "POST"):
def _e3():
raise RuntimeError("http.sse: метод %s запрещён" % method)
yield
return _e3()
body = params.get("json")
raw = params.get("body")
data = None
if body is not None:
import json as _j
data = _j.dumps(body, ensure_ascii=False).encode()
elif raw is not None:
data = raw.encode() if isinstance(raw, str) else bytes(raw)
headers = _headers(ep, secret, params)
if body is not None:
headers.setdefault("Content-Type", "application/json")
import time as _t
# Eager-подписка: соединение открывается ЗДЕСЬ (до первого next),
# иначе быстрые дельты прилетают раньше подписки и теряются.
import urllib3.exceptions as _ue
try:
r = pm.request(method, url, body=data, headers=headers,
timeout=timeout, retries=0, preload_content=False)
except Exception as e: # noqa: BLE001
raise RuntimeError("http.sse %s: %s" % (url.split("?")[0], e))
if r.status >= 400:
try:
body_txt = (r.read() or b"")[:2000].decode("utf-8", "replace")
except Exception: # noqa: BLE001
body_txt = ""
try:
r.release_conn()
except Exception: # noqa: BLE001
pass
raise RuntimeError("HTTP %d: %s" % (r.status, body_txt[:200]))
def gen():
buf = b""
total = 0
lines = 0
deadline = _t.monotonic() + timeout
try:
for chunk in r.stream(65536):
if not chunk:
continue
total += len(chunk)
if total > max_bytes:
raise RuntimeError("http.sse: лимит %d байт" % max_bytes)
if _t.monotonic() > deadline:
raise RuntimeError("http.sse: дедлайн %dс" % timeout)
buf += chunk
while b"\n" in buf:
line, buf = buf.split(b"\n", 1)
line = line.strip()
if not line.startswith(b"data:"):
continue
payload = line[5:].strip().decode("utf-8", "replace")
if not payload:
continue
lines += 1
if lines > max_lines:
raise RuntimeError("http.sse: лимит %d строк" % max_lines)
yield payload
except _ue.ReadTimeoutError:
raise RuntimeError("http.sse: таймаут чтения")
finally:
try:
r.release_conn()
except Exception: # noqa: BLE001
pass
return gen()
OPS["http.sse"] = sse_lines
def health(ep, secret):
hcfg = ep.get("health") or {}
hpath = (hcfg.get("health_path") or b.opt(ep, "health_path")
or "/").strip() or "/"
r = http_request(ep, secret, {"path": hpath, "method": "GET"})
if not r.get("ok"):
return r
expect = hcfg.get("expect_status", b.opt(ep, "expect_status", [200]))
try:
expect = [int(x) for x in expect]
except (TypeError, ValueError):
expect = [200]
if r.get("status") in expect:
return b.ok(ms=r.get("ms", 0), status=r.get("status"))
return b.err("http health: status=%s, ждали %s" % (r.get("status"),
expect))
drivers/mgmt.py 10.7 КБ
"""outbound.drivers.mgmt — winrm/docker/proxmox/telnet. Плюс dry-run.
winrm: pywinrm (опционально). docker: Engine API по unix-socket (stdlib
http.client) или TCP. proxmox: REST (ticket либо API-token) поверх urllib3.
telnet: сырые сокеты с expect (stdlib, без telnetlib). dry-run: заглушка.
"""
import json as _json
import socket as _socket
import time as _time
from . import base as b
KINDS = ("winrm", "docker", "proxmox", "telnet", "dryrun")
# --- winrm ---
def winrm_exec(ep, secret, params):
wr = b.lazy("winrm")
if not wr:
return b.missing("pywinrm", "winrm")
host = (b.addr(ep, "host") or "").strip()
user = (b.addr(ep, "user") or "").strip()
if not host or not user:
return b.err("winrm.exec: нужны address.host/user")
cmd = ((params or {}).get("cmd") or "").strip()
if not cmd:
return b.err("winrm.exec: нужен params.cmd")
scheme = (b.addr(ep, "scheme") or "http").strip()
try:
port = int(b.addr(ep, "port") or (5986 if scheme == "https" else 5985))
except (TypeError, ValueError):
port = 5985
t0 = b.now_ms()
try:
sess = wr.Session("%s://%s:%s/wsman" % (scheme, host, port),
auth=(user, secret or ""),
transport="ntlm",
server_cert_validation="ignore"
if b.opt(ep, "verify", True) is False else "validate")
r = sess.run_cmd(cmd)
out = b.ok(rc=r.status_code,
output=(r.std_out or b"").decode("utf-8", "replace"
)[:8000],
ms=b.now_ms() - t0)
if r.status_code != 0:
out["ok"] = False
out["error"] = "rc=%s: %s" % (
r.status_code,
(r.std_err or b"").decode("utf-8", "replace")[:300])
return out
except Exception as e: # noqa: BLE001
return b.err("winrm %s: %s" % (host, e), ms=b.now_ms() - t0)
# --- docker (Engine API) ---
def _docker_call(ep, method, path, body=None):
import http.client as _hc
sock = (b.addr(ep, "socket") or "").strip()
host = (b.addr(ep, "host") or "").strip()
t0 = b.now_ms()
try:
if sock:
conn = _hc.HTTPConnection("localhost", timeout=b.timeout(ep))
conn.sock = _socket.socket(_socket.AF_UNIX,
_socket.SOCK_STREAM)
conn.sock.settimeout(b.timeout(ep))
conn.sock.connect(sock)
elif host:
try:
port = int(b.addr(ep, "port") or 2375)
except (TypeError, ValueError):
port = 2375
conn = _hc.HTTPConnection(host, port, timeout=b.timeout(ep))
else:
return b.err("docker: нужны address.socket или address.host")
data = None
headers = {}
if body is not None:
data = _json.dumps(body)
headers["Content-Type"] = "application/json"
conn.request(method, "/v1.43" + path, body=data, headers=headers)
r = conn.getresponse()
text = r.read()[:100000].decode("utf-8", "replace")
try:
payload = _json.loads(text) if text else None
except Exception: # noqa: BLE001
payload = None
return b.ok(status=r.status, json=payload, ms=b.now_ms() - t0)
except Exception as e: # noqa: BLE001
return b.err("docker %s %s: %s" % (method, path, e))
def docker_ps(ep, secret, params):
all_c = (params or {}).get("all", True)
r = _docker_call(ep, "GET", "/containers/json?all=%s" % (
"1" if all_c else "0"))
if not r.get("ok"):
return r
items = [{"id": (c.get("Id") or "")[:12], "name": (c.get("Names")
or ["?"])[0].lstrip("/"),
"state": c.get("State"), "status": c.get("Status")}
for c in (r.get("json") or [])]
return b.ok(containers=items)
def docker_action(ep, secret, params):
name = ((params or {}).get("name") or "").strip()
act = ((params or {}).get("act") or "").strip()
if act not in ("start", "stop", "restart"):
return b.err("docker.action: act = start|stop|restart")
if not name:
return b.err("docker.action: нужен params.name")
return _docker_call(ep, "POST", "/containers/%s/%s" % (name, act))
# --- proxmox (REST: ticket или API-token) ---
def _pve(ep, secret, method, path, body=None):
from . import http as _h
user = (b.addr(ep, "user") or "").strip()
is_token = "!" in user # PVE API-токен: user@realm!tokenid
fake = {"opts": {"auth": "token" if is_token else "bearer",
"timeout": b.timeout(ep),
"verify": b.opt(ep, "verify", True),
"headers": {}},
"address": {"base_url": "https://%s:%s/api2/json" % (
b.addr(ep, "host"), b.addr(ep, "port") or 8006)}}
sec = secret
if not is_token:
# ticket-режим: сначала логин, токен в куку
login = _h.http_request(
{"opts": {"timeout": b.timeout(ep),
"verify": b.opt(ep, "verify", True)},
"address": fake["address"]},
None, {"path": "/access/ticket", "method": "POST",
"body": "username=%s&password=%s" % (user, secret or "")})
if not login.get("ok"):
return b.err("proxmox login: %s" % login.get("error"))
data = (login.get("json") or {}).get("data") or {}
ticket = data.get("ticket", "")
fake["opts"]["headers"] = {
"Cookie": "PVEAuthCookie=" + ticket,
"CSRFPreventionToken": data.get("CSRFPreventionToken", "")}
sec = None
else:
fake["opts"]["headers"] = {"Authorization": "PVEAPIToken=%s=%s" % (
user, secret or "")}
sec = None
params = {"path": path, "method": method}
if body is not None:
params["json"] = body
return _h.http_request(fake, sec, params)
def proxmox_get(ep, secret, params):
path = ((params or {}).get("path") or "").strip()
if not path.startswith("/"):
return b.err("proxmox.get: path с ведущим /")
return _pve(ep, secret, "GET", path)
def proxmox_action(ep, secret, params):
"""Действие с ВМ/контейнером: {node, type=qemu|lxc, vmid, act}."""
params = params or {}
node = (params.get("node") or "").strip()
vmid = str(params.get("vmid") or "").strip()
act = (params.get("act") or "").strip()
vtype = (params.get("type") or "qemu").strip()
if act not in ("start", "stop", "shutdown", "reboot"):
return b.err("proxmox.action: act=start|stop|shutdown|reboot")
if not node or not vmid:
return b.err("proxmox.action: нужны node и vmid")
return _pve(ep, secret, "POST", "/nodes/%s/%s/%s/status/%s" % (
node, vtype, vmid, act))
# --- telnet (сырые сокеты + expect) ---
def telnet_exec(ep, secret, params):
host = (b.addr(ep, "host") or "").strip()
try:
port = int(b.addr(ep, "port") or 23)
except (TypeError, ValueError):
port = 23
script = (params or {}).get("script") or []
if not host or not script:
return b.err("telnet.exec: нужны address.host и params.script "
"[{send|expect, ...}]")
user = (b.addr(ep, "user") or "").strip()
t0 = b.now_ms()
try:
s = _socket.create_connection((host, port), timeout=b.timeout(ep))
s.settimeout(b.timeout(ep))
buf = b""
transcript = []
def _read_until(marker, timeout_s):
nonlocal buf
end = _time.time() + timeout_s
want = marker.encode()
while _time.time() < end:
if want in buf:
idx = buf.index(want) + len(want)
chunk, buf = buf[:idx], buf[idx:]
return chunk
try:
data = s.recv(4096)
except Exception: # noqa: BLE001
break
if not data:
break
buf += _telnet_strip(data)
chunk, buf = buf, b""
return chunk
_read_until("ogin:", 8)
if user:
s.sendall((user + "\n").encode())
_read_until("assword:", 8)
if secret:
s.sendall((secret + "\n").encode())
_read_until("#", 8)
_read_until(">", 3)
for step in script:
if "expect" in step:
chunk = _read_until(step["expect"], b.timeout(ep))
transcript.append(chunk.decode("utf-8",
"replace")[-2000:])
elif "send" in step:
s.sendall((step["send"] + "\n").encode())
_time.sleep(0.3)
try:
s.sendall(b"exit\n")
except Exception: # noqa: BLE001
pass
s.close()
return b.ok(transcript="\n".join(transcript)[-8000:],
ms=b.now_ms() - t0)
except Exception as e: # noqa: BLE001
return b.err("telnet %s: %s" % (host, e))
def _telnet_strip(data):
"""Вырезать IAC-последовательности."""
out = bytearray()
i = 0
while i < len(data):
if data[i] == 0xFF and i + 2 < len(data):
i += 3
else:
out.append(data[i])
i += 1
return bytes(out)
# --- dry-run ---
def dryrun_call(ep, secret, params):
return b.ok(dry_run=True, endpoint=ep.get("name"),
echo=(params or {}))
OPS = {
"winrm.exec": winrm_exec,
"docker.ps": docker_ps,
"docker.action": docker_action,
"proxmox.get": proxmox_get,
"proxmox.action": proxmox_action,
"telnet.exec": telnet_exec,
"dryrun.call": dryrun_call,
}
def health(ep, secret):
kind = (ep.get("kind") or "").strip()
if kind == "docker":
r = _docker_call(ep, "GET", "/version")
return b.ok(version=((r.get("json") or {}).get("Version", "?"))
) if r.get("ok") else r
if kind == "proxmox":
r = _pve(ep, secret, "GET", "/version")
return b.ok() if r.get("ok") else r
if kind == "winrm":
r = winrm_exec(ep, secret, {"cmd": "echo outbound-ok"})
return b.ok() if r.get("ok") else r
if kind == "telnet":
from . import net as _n
host = (b.addr(ep, "host") or "").strip()
return _n.tcp_open({"address": {"host": host,
"port": b.addr(ep, "port") or 23},
"opts": ep.get("opts")}, None, {})
if kind == "dryrun":
return b.ok(dry_run=True)
return b.err("mgmt: нет health для kind=%s" % kind)
drivers/msg.py 8.5 КБ
"""outbound.drivers.msg — mqtt (paho, опционально) и ws (свой мини-клиент stdlib).
mqtt: publish + subscribe-разовый (слушаем N секунд, собираем сообщения).
ws: текстовые фреймы (RFC 6455, без extensions), wss через ssl.
"""
import socket as _socket
import ssl as _ssl
from . import base as b
KINDS = ("mqtt", "ws")
# --- mqtt ---
def _mqtt_client(ep, secret):
paho = b.lazy("paho.mqtt.client")
if not paho:
return None, b.missing("paho-mqtt", "mqtt")
user = (b.addr(ep, "user") or "").strip()
try:
cli = paho.Client()
except Exception as e: # noqa: BLE001
return None, b.err("mqtt: %s" % e)
if user:
cli.username_pw_set(user, secret or None)
tls = b.opt(ep, "tls", False)
if tls:
try:
cli.tls_set()
except Exception as e: # noqa: BLE001
return None, b.err("mqtt tls: %s" % e)
return cli, None
def _mqtt_addr(ep):
host = (b.addr(ep, "host") or "").strip()
try:
port = int(b.addr(ep, "port") or 1883)
except (TypeError, ValueError):
port = 1883
return host, port
def mqtt_publish(ep, secret, params):
params = params or {}
topic = (params.get("topic") or "").strip()
if not topic:
return b.err("mqtt.publish: нужен params.topic")
payload = params.get("payload", "")
if not isinstance(payload, str):
import json as _j
payload = _j.dumps(payload, ensure_ascii=False)
try:
qos = int(params.get("qos", 0))
except (TypeError, ValueError):
qos = 0
host, port = _mqtt_addr(ep)
cli, err = _mqtt_client(ep, secret)
if err:
return err
t0 = b.now_ms()
try:
cli.connect(host, port, keepalive=30)
info = cli.publish(topic, payload, qos=max(0, min(qos, 2)))
info.wait_for_publish(timeout=b.timeout(ep))
cli.disconnect()
return b.ok(topic=topic, ms=b.now_ms() - t0)
except Exception as e: # noqa: BLE001
try:
cli.disconnect()
except Exception: # noqa: BLE001
pass
return b.err("mqtt.publish %s: %s" % (topic, e))
def mqtt_subscribe(ep, secret, params):
params = params or {}
topic = (params.get("topic") or "").strip()
if not topic:
return b.err("mqtt.subscribe: нужен params.topic")
try:
wait_s = int(params.get("wait", 10))
except (TypeError, ValueError):
wait_s = 10
wait_s = max(1, min(wait_s, 120))
host, port = _mqtt_addr(ep)
cli, err = _mqtt_client(ep, secret)
if err:
return err
import time as _t
got = []
def _on_msg(_c, _u, msg):
try:
got.append({"topic": msg.topic,
"payload": msg.payload.decode("utf-8", "replace")})
except Exception: # noqa: BLE001
pass
t0 = b.now_ms()
try:
cli.on_message = _on_msg
cli.connect(host, port, keepalive=30)
cli.subscribe(topic)
cli.loop_start()
_t.sleep(wait_s)
cli.loop_stop()
cli.disconnect()
return b.ok(topic=topic, messages=got[:50],
count=len(got), ms=b.now_ms() - t0)
except Exception as e: # noqa: BLE001
try:
cli.loop_stop()
cli.disconnect()
except Exception: # noqa: BLE001
pass
return b.err("mqtt.subscribe %s: %s" % (topic, e))
# --- ws (мини-клиент, текстовые фреймы) ---
def _ws_key():
import base64 as _b64
import os as _o
return _b64.b64encode(_o.urandom(16)).decode()
def _ws_connect(ep, secret, params):
url = ((params or {}).get("url") or b.addr(ep, "base_url")
or "").strip()
if not url.startswith(("ws://", "wss://")):
return None, b.err("ws: нужен ws:// или wss:// url")
secure = url.startswith("wss://")
rest = url[5:] if secure else url[4:]
hostport, _, path = rest.partition("/")
path = "/" + path
if ":" in hostport:
host, _, port = hostport.partition(":")
port = int(port)
else:
host, port = hostport, 443 if secure else 80
key = _ws_key()
req = ("GET %s HTTP/1.1\r\nHost: %s\r\nUpgrade: websocket\r\n"
"Connection: Upgrade\r\nSec-WebSocket-Key: %s\r\n"
"Sec-WebSocket-Version: 13\r\n" % (path, host, key))
token = (b.opt(ep, "auth") or "").strip().lower()
if token == "bearer" and secret:
req += "Authorization: Bearer %s\r\n" % secret
req += "\r\n"
t0 = b.now_ms()
try:
s = _socket.create_connection((host, port), timeout=b.timeout(ep))
if secure:
ctx = _ssl.create_default_context()
if b.opt(ep, "verify", True) is False:
ctx.check_hostname = False
ctx.verify_mode = _ssl.CERT_NONE
s = ctx.wrap_socket(s, server_hostname=host)
s.sendall(req.encode())
resp = b""
while b"\r\n\r\n" not in resp:
chunk = s.recv(4096)
if not chunk:
break
resp += chunk
if b" 101 " not in resp.split(b"\r\n", 1)[0]:
s.close()
return None, b.err("ws: handshake fail: %s" %
resp[:120].decode("utf-8", "replace"))
return (s, b.now_ms() - t0), None
except Exception as e: # noqa: BLE001
return None, b.err("ws.connect: %s" % e)
def _ws_send_text(s, text):
import os as _o
import struct as _st
data = text.encode("utf-8")
mask = _o.urandom(4)
hdr = bytes([0x81])
ln = len(data)
if ln < 126:
hdr += _st.pack("!B", 0x80 | ln)
elif ln < 65536:
hdr += _st.pack("!BH", 0x80 | 126, ln)
else:
hdr += _st.pack("!BQ", 0x80 | 127, ln)
masked = bytes(c ^ mask[i % 4] for i, c in enumerate(data))
s.sendall(hdr + mask + masked)
def _ws_recv(s, timeout_s):
import struct as _st
s.settimeout(timeout_s)
try:
hdr = s.recv(2)
if len(hdr) < 2:
return None
ln = hdr[1] & 0x7F
if ln == 126:
ln = _st.unpack("!H", s.recv(2))[0]
elif ln == 127:
ln = _st.unpack("!Q", s.recv(8))[0]
data = b""
while len(data) < ln:
chunk = s.recv(ln - len(data))
if not chunk:
break
data += chunk
opcode = hdr[0] & 0x0F
if opcode == 0x8:
return None
return data.decode("utf-8", "replace")
except Exception: # noqa: BLE001
return None
def ws_roundtrip(ep, secret, params):
"""Отправить текст, собрать до N ответов (wait_s)."""
params = params or {}
text = (params.get("send") or "")
if not isinstance(text, str):
return b.err("ws.roundtrip: нужен params.send (текст)")
try:
wait_s = int(params.get("wait", 5))
except (TypeError, ValueError):
wait_s = 5
wait_s = max(1, min(wait_s, 60))
(conn, ms0), err = _ws_connect(ep, secret, params)
if err:
return err
s = conn
import time as _t
t0 = b.now_ms()
try:
_ws_send_text(s, text)
got = []
deadline = _t.time() + wait_s
while _t.time() < deadline and len(got) < 20:
msg = _ws_recv(s, max(0.5, deadline - _t.time()))
if msg is None:
break
got.append(msg[:4000])
s.close()
return b.ok(messages=got, count=len(got), connect_ms=ms0,
ms=b.now_ms() - t0)
except Exception as e: # noqa: BLE001
try:
s.close()
except Exception: # noqa: BLE001
pass
return b.err("ws.roundtrip: %s" % e)
OPS = {
"mqtt.publish": mqtt_publish,
"mqtt.subscribe": mqtt_subscribe,
"ws.roundtrip": ws_roundtrip,
}
def health(ep, secret):
kind = (ep.get("kind") or "").strip()
if kind == "mqtt":
host, port = _mqtt_addr(ep)
cli, err = _mqtt_client(ep, secret)
if err:
return err
t0 = b.now_ms()
try:
cli.connect(host, port, keepalive=10)
cli.disconnect()
return b.ok(ms=b.now_ms() - t0)
except Exception as e: # noqa: BLE001
return b.err("mqtt health: %s" % e)
if kind == "ws":
(conn, ms0), err = _ws_connect(ep, secret, {})
if err:
return err
try:
conn.close()
except Exception: # noqa: BLE001
pass
return b.ok(ms=ms0)
return b.err("msg: нет health для kind=%s" % kind)
drivers/net.py 7.8 КБ
"""outbound.drivers.net — L3-L4: tcp/udp/icmp/dns/wol. Только stdlib."""
import socket
import struct
import time
from . import base as b
KINDS = ("tcp", "udp", "icmp", "dns", "wol")
def _target(ep):
host = (b.addr(ep, "host") or "").strip()
try:
port = int(b.addr(ep, "port") or 0)
except (TypeError, ValueError):
port = 0
return host, port
# --- tcp ---
def tcp_open(ep, secret, params):
host, port = _target(ep)
if not host or not port:
return b.err("tcp.open: нужны address.host и address.port")
t0 = b.now_ms()
try:
s = socket.create_connection((host, port),
timeout=b.timeout(ep))
s.close()
return b.ok(ms=b.now_ms() - t0, host=host, port=port)
except Exception as e: # noqa: BLE001
return b.err("tcp %s:%s: %s" % (host, port, e),
ms=b.now_ms() - t0)
def tcp_banner(ep, secret, params):
"""Открыть, прочитать до N байт (баннер), закрыть."""
host, port = _target(ep)
try:
n = int((params or {}).get("bytes", 256))
except (TypeError, ValueError):
n = 256
n = max(1, min(n, 4096))
t0 = b.now_ms()
try:
s = socket.create_connection((host, port), timeout=b.timeout(ep))
s.settimeout(b.timeout(ep))
data = s.recv(n)
s.close()
return b.ok(banner=data.decode("utf-8", "replace"),
ms=b.now_ms() - t0)
except Exception as e: # noqa: BLE001
return b.err("tcp.banner %s:%s: %s" % (host, port, e))
# --- udp ---
def udp_send(ep, secret, params):
host, port = _target(ep)
payload = ((params or {}).get("data") or "")
if isinstance(payload, str):
payload = payload.encode("utf-8")
t0 = b.now_ms()
try:
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.settimeout(b.timeout(ep))
s.sendto(bytes(payload), (host, port))
s.close()
return b.ok(sent=len(bytes(payload)), ms=b.now_ms() - t0)
except Exception as e: # noqa: BLE001
return b.err("udp %s:%s: %s" % (host, port, e))
# --- icmp (через системный ping, без root) ---
def icmp_ping(ep, secret, params):
host, _ = _target(ep)
if not host:
return b.err("icmp.ping: нужен address.host")
try:
count = int((params or {}).get("count", 2))
except (TypeError, ValueError):
count = 2
count = max(1, min(count, 5))
t0 = b.now_ms()
rc, out, err = b.run(["ping", "-c", str(count), "-W", "2", host],
timeout_s=b.timeout(ep) + 5)
if rc == 0:
return b.ok(ms=b.now_ms() - t0, detail=_ping_summary(out))
return b.err("ping %s: %s" % (host, (err or out or "rc=%s" % rc)[:200]),
ms=b.now_ms() - t0)
def _ping_summary(out):
for line in (out or "").splitlines():
if "min/avg/max" in line:
return line.strip()[:200]
return ""
# --- dns (резолв + запрос записей по UDP) ---
def dns_resolve(ep, secret, params):
name = ((params or {}).get("name") or b.addr(ep, "host") or "").strip()
if not name:
return b.err("dns.resolve: нужны params.name или address.host")
t0 = b.now_ms()
try:
infos = socket.getaddrinfo(name, None, socket.AF_UNSPEC,
socket.SOCK_STREAM)
addrs = sorted({i[4][0] for i in infos})
return b.ok(name=name, addresses=addrs, ms=b.now_ms() - t0)
except Exception as e: # noqa: BLE001
return b.err("dns %s: %s" % (name, e))
def dns_query(ep, secret, params):
"""Запрос записи типа A/AAAA/MX/TXT к address.host (DNS-сервер)."""
server = (b.addr(ep, "host") or "77.88.8.8").strip()
try:
port = int(b.addr(ep, "port") or 53)
except (TypeError, ValueError):
port = 53
name = ((params or {}).get("name") or "").strip()
rtype = ((params or {}).get("type") or "A").strip().upper()
if not name:
return b.err("dns.query: нужен params.name")
pkt = _dns_packet(name, rtype)
if pkt is None:
return b.err("dns.query: тип %s не поддерживается" % rtype)
t0 = b.now_ms()
try:
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.settimeout(b.timeout(ep))
s.sendto(pkt, (server, port))
data, _ = s.recvfrom(2048)
s.close()
return b.ok(name=name, type=rtype, answers=_dns_parse(data),
ms=b.now_ms() - t0)
except Exception as e: # noqa: BLE001
return b.err("dns.query %s %s: %s" % (name, rtype, e))
_TYPEMAP = {"A": 1, "AAAA": 28, "MX": 15, "TXT": 16, "CNAME": 5}
def _dns_packet(name, rtype):
import random as _r
qtype = _TYPEMAP.get(rtype)
if not qtype:
return None
hdr = struct.pack(">HHHHHH", _r.getrandbits(16), 0x0100, 1, 0, 0, 0)
qname = b"".join(bytes([len(p)]) + p.encode("ascii", "ignore")
for p in name.split(".") if p) + b"\x00"
return hdr + qname + struct.pack(">HH", qtype, 1)
def _dns_parse(data):
"""Минимум: вытащить A/AAAA/CNAME/TXT из answer-секции."""
out = []
try:
ancount = struct.unpack(">H", data[6:8])[0]
pos = 12
while data[pos] != 0: # пропуск question
pos += data[pos] + 1
pos += 5
for _ in range(min(ancount, 10)):
while pos < len(data) and data[pos] >= 0xC0:
pos += 2
break
else:
while pos < len(data) and data[pos] != 0:
pos += data[pos] + 1
pos += 1
if pos + 10 > len(data):
break
rtype, _, _, rdlen = struct.unpack(">HHIH", data[pos:pos + 10])
pos += 10
rdata = data[pos:pos + rdlen]
pos += rdlen
if rtype == 1 and rdlen == 4:
out.append(socket.inet_ntoa(rdata))
elif rtype == 28 and rdlen == 16:
out.append(socket.inet_ntop(socket.AF_INET6, rdata))
elif rtype in (5,):
out.append("cname")
elif rtype == 16:
out.append(rdata[1:].decode("utf-8", "replace"))
except Exception: # noqa: BLE001
pass
return out
# --- wol ---
def wol_send(ep, secret, params):
mac = ((params or {}).get("mac") or b.addr(ep, "mac") or "")
mac = "".join(c for c in mac if c.isalnum()).lower()
if len(mac) != 12:
return b.err("wol.send: нужен MAC (params.mac или address.mac)")
bcast = (b.addr(ep, "broadcast") or "255.255.255.255").strip()
try:
port = int(b.addr(ep, "port") or 9)
except (TypeError, ValueError):
port = 9
pkt = b"\xff" * 6 + bytes.fromhex(mac) * 16
try:
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
s.sendto(pkt, (bcast, port))
s.close()
return b.ok(mac=mac, via="%s:%s" % (bcast, port))
except Exception as e: # noqa: BLE001
return b.err("wol: %s" % e)
OPS = {
"tcp.open": tcp_open,
"tcp.banner": tcp_banner,
"udp.send": udp_send,
"icmp.ping": icmp_ping,
"dns.resolve": dns_resolve,
"dns.query": dns_query,
"wol.send": wol_send,
}
def health(ep, secret):
kind = (ep.get("kind") or "").strip()
if kind == "tcp":
return tcp_open(ep, secret, {})
if kind in ("udp", "wol"):
host, _ = _target(ep)
return icmp_ping({"address": {"host": host or "127.0.0.1"},
"opts": ep.get("opts")}, secret, {"count": 1})
if kind == "dns":
return dns_query(ep, secret, {"name": "ya.ru", "type": "A"})
if kind == "icmp":
return icmp_ping(ep, secret, {"count": 2})
return b.err("net: нет health для kind=%s" % kind)
drivers/ssh.py 3.6 КБ
"""outbound.drivers.ssh — выполнение команд по SSH через бинарь OpenSSH.
Аутентификация: address.user + secret как пароль (через SSHPASS+sshpass,
как в services_windows) либо address.key (путь к ключу) без секрета.
Сквозной режим via: имя другой endpoint-точки kind=ssh — ProxyJump (-J).
Шелл не используется в принципе: команда уходит единым argv.
"""
import os as _os
from . import base as b
KINDS = ("ssh",)
def _base_cmd(ep, secret, jump=None):
user = (b.addr(ep, "user") or "").strip()
host = (b.addr(ep, "host") or "").strip()
if not host:
return None, b.err("ssh.exec: нужен address.host")
try:
port = int(b.addr(ep, "port") or 22)
except (TypeError, ValueError):
port = 22
key = (b.addr(ep, "key") or "").strip()
target = "%s@%s" % (user, host) if user else host
cmd = ["ssh", "-p", str(port), "-o", "BatchMode=%s" % (
"yes" if key else "no"),
"-o", "StrictHostKeyChecking=%s" % b.opt(
ep, "strict_host_key", "accept-new"),
"-o", "ConnectTimeout=%s" % min(b.timeout(ep), 30)]
if jump:
cmd += ["-J", jump]
if key:
cmd += ["-i", key]
cmd.append(target)
return cmd, ("" if key else (secret or ""))
def ssh_exec(ep, secret, params, jump=None):
params = params or {}
command = (params.get("cmd") or "").strip()
if not command:
return b.err("ssh.exec: нужен params.cmd")
if len(command) > 8000:
return b.err("ssh.exec: команда длиннее 8000 символов")
stdin_text = params.get("input")
if stdin_text is not None:
stdin_text = str(stdin_text)
if len(stdin_text) > 32768:
return b.err("ssh.exec: stdin длиннее 32K")
try:
max_chars = int(params.get("max_chars") or 0)
except (TypeError, ValueError):
max_chars = 0
if max_chars:
max_chars = max(1, min(max_chars, 8000000))
else:
max_chars = 8000
try:
timeout = int(params.get("timeout") or 0)
except (TypeError, ValueError):
timeout = 0
if timeout:
timeout = max(5, min(timeout, 300))
else:
timeout = b.timeout(ep) + 10
cmd, password = _base_cmd(ep, secret, jump)
if cmd is None: # ошибка из _base_cmd лежит в password
return password
env = None
full = list(cmd)
if password:
full = ["sshpass", "-e"] + full
env = dict(_os.environ)
env["SSHPASS"] = password
full.append(command)
t0 = b.now_ms()
rc, out, err = b.run(full, timeout_s=timeout, env=env,
input_text=stdin_text)
if rc is None:
return b.err("ssh %s: %s" % (b.addr(ep, "host"), err),
ms=b.now_ms() - t0)
res = b.ok(rc=rc, output=out.strip()[:max_chars], ms=b.now_ms() - t0)
if len(out.strip()) > max_chars:
res["truncated"] = True
if rc != 0:
res["ok"] = False
res["error"] = "rc=%s: %s" % (rc, (err or out)[:300])
return res
OPS = {
"ssh.exec": ssh_exec,
}
def health(ep, secret, jump=None):
hcfg = ep.get("health") or {}
probe = (hcfg.get("health_cmd") or b.opt(ep, "health_cmd")
or "echo outbound-ok").strip()
r = ssh_exec(ep, secret, {"cmd": probe}, jump=jump)
if r.get("ok") and "outbound-ok" in (r.get("output") or ""):
return b.ok(ms=r.get("ms", 0))
if r.get("ok"):
return b.ok(ms=r.get("ms", 0))
return b.err("ssh health: %s" % r.get("error", "?"))
module.py 17.1 КБ
"""outbound.Module — шлюз внешних подключений.
Сущности: точки (endpoints.json), потребители+привязки (bindings.json),
секреты — только .env модуля (auth_ref). Вызовы — через outbound.call
с обязательной привязкой; без неё — понятная ошибка в кабинет.
Health всех точек — фоновым тиком в ctx.cache.ns("outbound").
"""
import logging
import os
import threading
import time
from core.base_module import BaseModule
from . import drivers
from .store import Store
logger = logging.getLogger(__name__)
class Module(BaseModule):
def setup(self, ctx) -> None:
mod_dir = os.path.dirname(os.path.abspath(__file__))
self.config = ctx.module_config("outbound", mod_dir)
self.ctx = ctx
self.store = Store(ctx)
try:
self._log_limit = int((self.config.get("LOG_LIMIT")
if self.config else None) or 200)
except (TypeError, ValueError):
self._log_limit = 200
try:
self._health_interval = int((self.config.get("HEALTH_INTERVAL")
if self.config else None) or 60)
except (TypeError, ValueError):
self._health_interval = 60
self._log = [] # кольцевой журнал в памяти
self._stop = threading.Event()
self._thread = None
api = ctx.api.register
api("outbound", "endpoint_list", self.api_endpoint_list)
api("outbound", "endpoint_add", self.api_endpoint_add)
api("outbound", "endpoint_update", self.api_endpoint_update)
api("outbound", "endpoint_delete", self.api_endpoint_delete)
api("outbound", "endpoint_test", self.api_endpoint_test)
api("outbound", "consumer_list", self.api_consumer_list)
api("outbound", "consumer_ensure", self.api_consumer_ensure)
api("outbound", "bind", self.api_bind)
api("outbound", "unbind", self.api_unbind)
api("outbound", "call", self.api_call)
api("outbound", "health", self.api_health)
api("outbound", "log", self.api_log)
api("outbound", "kinds", lambda: drivers.kinds())
api("outbound", "endpoint_base", self.api_endpoint_base)
def start(self) -> None:
if self._thread is None:
self._thread = threading.Thread(target=self._health_loop,
name="outbound-health",
daemon=True)
self._thread.start()
def stop(self) -> None:
self._stop.set()
# --- внутреннее ---
def _secret(self, ep):
ref = (ep.get("auth_ref") or "").strip()
if not ref:
return ""
try:
val = self.config.get(ref, "") if self.config else ""
except Exception: # noqa: BLE001
val = ""
return val or ""
def _public_ep(self, name, ep):
return {"name": name, "kind": ep.get("kind"),
"address": _mask_addr(ep.get("address") or {}),
"opts": ep.get("opts") or {},
"auth_ref": bool(ep.get("auth_ref")),
"via": ep.get("via") or "", "proxy": bool(ep.get("proxy")),
"health_cfg": ep.get("health") or {},
"note": ep.get("note") or ""}
def _via_jump(self, ep, eps, depth=0):
"""ProxyJump-цель для ssh-точки: user@host jump-точки."""
via = (ep.get("via") or "").strip()
if not via or depth > 2:
return ""
jump = (eps or {}).get(via)
if not jump or (jump.get("kind") or "") != "ssh":
raise ValueError("via: %s не ssh-точка" % via)
user = (jump.get("address") or {}).get("user", "")
host = (jump.get("address") or {}).get("host", "")
if not host:
raise ValueError("via: у %s нет host" % via)
if jump.get("via"):
raise ValueError("via: цепочки длиннее 1 запрещены")
return "%s@%s" % (user, host) if user else host
def _health_loop(self):
while not self._stop.wait(self._health_interval):
try:
self._poll_all()
except Exception: # noqa: BLE001
logger.exception("outbound: health-тик")
def _poll_all(self):
eps = self.store.endpoints()
if not eps:
return
try:
ns = self.ctx.cache.ns("outbound")
except Exception: # noqa: BLE001
return
for name, ep in eps.items():
if not (ep.get("health") or {}).get("enabled", True):
continue
try:
jump = self._via_jump(ep, eps)
except ValueError:
jump = None
if jump is None and ep.get("via"):
res = {"ok": False, "error": "битый via"}
else:
res = drivers.check_health(ep, self._secret(ep),
via_jump=jump or None)
res["at"] = int(time.time())
try:
ns.set("health:%s" % name, res)
except Exception: # noqa: BLE001
pass
def _append_log(self, entry):
self._log.append(entry)
if len(self._log) > self._log_limit:
del self._log[:-self._log_limit]
# --- API: точки ---
def api_endpoint_list(self):
eps = self.store.endpoints()
return [self._public_ep(n, e) for n, e in sorted(eps.items())]
def _validate_ep(self, name, spec):
if not name or len(name) > 64:
raise ValueError("имя точки: 1..64 символа")
if any(c not in "abcdefghijklmnopqrstuvwxyz0123456789-_"
for c in name.lower()):
raise ValueError("имя точки: латиница/цифры/-/_")
kind = (spec.get("kind") or "").strip()
if kind not in drivers.kinds():
raise ValueError("kind: %s (доступны: %s)" % (
kind, ", ".join(drivers.kinds())))
address = spec.get("address") or {}
if not isinstance(address, dict):
raise ValueError("address — объект")
via = (spec.get("via") or "").strip()
if via and via == name:
raise ValueError("via на саму себя запрещён")
return {"kind": kind, "address": address,
"opts": spec.get("opts") or {},
"auth_ref": (spec.get("auth_ref") or "").strip(),
"via": via, "proxy": (spec.get("proxy") or "").strip(),
"health": spec.get("health") or {},
"note": (spec.get("note") or "")[:500]}
def api_endpoint_add(self, name, kind=None, address=None, **kw):
name = (name or "").strip()
spec = dict(kw)
if kind is not None:
spec["kind"] = kind
if address is not None:
spec["address"] = address
eps = self.store.endpoints()
if name in eps:
raise ValueError("точка %s уже есть" % name)
ep = self._validate_ep(name, spec)
if ep["via"] and ep["via"] not in eps:
raise ValueError("via: точки %s нет" % ep["via"])
# пробное подключение ДО сохранения (dryrun всегда ок)
probe = drivers.check_health(dict(ep, name=name),
self._secret(ep))
if not probe.get("ok"):
raise ValueError("точка не отвечает: %s" % probe.get("error"))
eps[name] = ep
self.store.save_endpoints(eps)
logger.info("outbound: точка %s (%s) добавлена", name, ep["kind"])
return {"ok": True, "name": name}
def api_endpoint_update(self, name, patch=None):
name = (name or "").strip()
eps = self.store.endpoints()
if name not in eps:
raise ValueError("точки %s нет" % name)
merged = dict(eps[name])
merged.update(patch or {})
eps[name] = self._validate_ep(name, merged)
self.store.save_endpoints(eps)
return {"ok": True}
def api_endpoint_delete(self, name):
name = (name or "").strip()
eps = self.store.endpoints()
if name not in eps:
raise ValueError("точки %s нет" % name)
del eps[name]
self.store.save_endpoints(eps)
data = self.store.bindings()
for mod, bound in list((data.get("bindings") or {}).items()):
if bound == name:
data["bindings"][mod] = None
self.store.save_bindings(data)
return {"ok": True}
def api_endpoint_test(self, name):
name = (name or "").strip()
eps = self.store.endpoints()
ep = eps.get(name)
if ep is None:
raise ValueError("точки %s нет" % name)
try:
jump = self._via_jump(ep, eps)
except ValueError as e:
return {"ok": False, "error": str(e)}
res = drivers.check_health(dict(ep, name=name),
self._secret(ep),
via_jump=jump or None)
try:
ns = self.ctx.cache.ns("outbound")
res["at"] = int(time.time())
ns.set("health:%s" % name, res)
except Exception: # noqa: BLE001
pass
return res
# --- API: потребители и привязки ---
def api_consumer_ensure(self, module, need=None, desc=None):
module = (module or "").strip()
if not module:
raise ValueError("consumer_ensure: пустой module")
data = self.store.bindings()
consumers = data.setdefault("consumers", {})
cur = consumers.get(module) or {}
cur.update({"need": (need or cur.get("need") or "").strip(),
"desc": (desc or cur.get("desc") or "")[:300]})
consumers[module] = cur
self.store.save_bindings(data)
bound = (data.get("bindings") or {}).get(module)
return {"ok": True, "bound": bound}
def api_consumer_list(self):
data = self.store.bindings()
bindings = data.get("bindings") or {}
consumers = data.get("consumers") or {}
out = []
for mod in sorted(set(list(bindings) + list(consumers))):
info = consumers.get(mod) or {}
out.append({"module": mod, "need": info.get("need", ""),
"desc": info.get("desc", ""),
"bound": bindings.get(mod)})
return out
def api_bind(self, module, endpoint=None, actor=""):
module = (module or "").strip()
endpoint = (endpoint or "").strip() or None
data = self.store.bindings()
if endpoint is not None and endpoint not in self.store.endpoints():
raise ValueError("точки %s нет" % endpoint)
data.setdefault("bindings", {})[module] = endpoint
data.setdefault("consumers", {}).setdefault(module, {})
self.store.save_bindings(data)
logger.info("outbound: %s -> %s", module, endpoint)
try:
self.ctx.api.call("audit.record", actor or "?",
"outbound.bind", module,
"-> %s" % (endpoint or "null"), True)
except Exception: # noqa: BLE001
pass
return {"ok": True}
def api_unbind(self, module):
return self.api_bind(module, None)
# --- API: единый вызов ---
def api_call(self, consumer, op, params=None):
import time as _t
consumer = (consumer or "").strip()
op = (op or "").strip()
t0 = _t.time()
data = self.store.bindings()
bound = (data.get("bindings") or {}).get(consumer)
if not bound:
self.api_consumer_ensure(consumer, op, "")
err = {"ok": False,
"error": "модуль %s не привязан к точке "
"(кабинет outbound → Клиенты)" % consumer}
self._append_log({"at": int(t0), "consumer": consumer,
"endpoint": None, "op": op,
"ms": 0, "ok": False,
"detail": "unbound"})
return err
eps = self.store.endpoints()
ep = eps.get(bound)
if ep is None:
return {"ok": False, "error": "точка %s удалена" % bound}
try:
jump = self._via_jump(ep, eps)
except ValueError as e:
return {"ok": False, "error": str(e)}
res = drivers.dispatch(dict(ep, name=bound), self._secret(ep),
op, params or {}, via_jump=jump or None)
if hasattr(res, "__next__"):
return self._wrap_stream(consumer, bound, op, res, t0)
ms = int((_t.time() - t0) * 1000)
self._append_log({"at": int(t0), "consumer": consumer,
"endpoint": bound, "op": op, "ms": ms,
"ok": bool(res.get("ok")),
"detail": ("" if res.get("ok")
else res.get("error", ""))[:200]})
return res
# --- API: health и журнал ---
def _wrap_stream(self, consumer, endpoint, op, gen, t0):
"""Учёт генератора: строки, мс, итог — в журнал по закрытии."""
import time as _t
count = [0]
def wrapped():
try:
for item in gen:
count[0] += 1
yield item
except GeneratorExit:
raise
except Exception as e: # noqa: BLE001
self._append_log({"at": int(t0), "consumer": consumer,
"endpoint": endpoint, "op": op,
"ms": int((_t.time() - t0) * 1000),
"ok": False,
"detail": ("stream: %s" % e)[:200]})
raise
else:
self._append_log({"at": int(t0), "consumer": consumer,
"endpoint": endpoint, "op": op,
"ms": int((_t.time() - t0) * 1000),
"ok": True,
"detail": "lines=%d" % count[0]})
finally:
try:
gen.close()
except Exception: # noqa: BLE001
pass
return wrapped()
def api_health(self):
try:
ns = self.ctx.cache.ns("outbound")
except Exception: # noqa: BLE001
return {}
out = {}
for name in self.store.endpoints():
try:
out[name] = ns.get("health:%s" % name) or {"ok": None,
"error": "ещё не проверялась"}
except Exception: # noqa: BLE001
out[name] = {"ok": None}
return out
def api_log(self, limit=None):
try:
limit = int(limit or 50)
except (TypeError, ValueError):
limit = 50
return list(self._log[-max(1, min(limit, self._log_limit)):])
def api_endpoint_base(self, consumer, field="base_url"):
"""Значение поля адреса привязанной точки (напр. base_url).
Единственный источник адреса для потребителей: base_url/host/port
живут только в точке, не в конфигах модулей.
"""
consumer = (consumer or "").strip()
field = (field or "").strip()
if field not in ("base_url", "host", "port", "user"):
raise ValueError("поле: base_url|host|port|user")
data = self.store.bindings()
bound = (data.get("bindings") or {}).get(consumer)
if not bound:
raise ValueError(
"модуль %s не привязан к точке "
"(кабинет outbound → Клиенты)" % consumer)
ep = self.store.endpoints().get(bound)
if ep is None:
raise ValueError("точка %s удалена" % bound)
val = (ep.get("address") or {}).get(field, "")
if val in (None, ""):
raise ValueError("точка %s без %s" % (bound, field))
return val
def health(self) -> dict:
try:
n_ep = len(self.store.endpoints())
data = self.store.bindings()
n_bound = sum(1 for v in (data.get("bindings") or {}).values()
if v)
return {"ok": True, "module": self.name,
"endpoints": n_ep, "bound": n_bound}
except Exception as e: # noqa: BLE001
return {"ok": False, "module": self.name, "error": str(e)}
def _mask_addr(address):
out = dict(address)
for key in ("password", "passwd", "token", "secret"):
if key in out:
out[key] = "***"
return out
store.py 1.8 КБ
"""outbound.store — endpoints.json / bindings.json в data/outbound/.
Секретов здесь нет: только auth_ref (имя переменной в .env модуля).
"""
import json
import logging
import os
logger = logging.getLogger(__name__)
_ENDPOINTS_FILE = "endpoints.json"
_BINDINGS_FILE = "bindings.json"
def data_subdir(ctx):
path = os.path.join(ctx.data_dir, "outbound")
os.makedirs(path, mode=0o700, exist_ok=True)
return path
def _load(path, default):
try:
with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
return data if isinstance(data, default.__class__) else default
except FileNotFoundError:
return default
except Exception as e: # noqa: BLE001
logger.warning("outbound.store: %s битый (%s), беру пустой",
path, e)
return default
def _save(path, data):
tmp = path + ".tmp"
with open(tmp, "w", encoding="utf-8") as f:
json.dump(data, f, ensure_ascii=False, indent=2)
os.replace(tmp, path)
try:
os.chmod(path, 0o600)
except Exception: # noqa: BLE001
pass
class Store:
def __init__(self, ctx):
self._dir = data_subdir(ctx)
self._ep_path = os.path.join(self._dir, _ENDPOINTS_FILE)
self._bd_path = os.path.join(self._dir, _BINDINGS_FILE)
# --- endpoints: {name: {...}} ---
def endpoints(self):
return _load(self._ep_path, {})
def save_endpoints(self, eps):
_save(self._ep_path, eps)
# --- bindings: {module: endpoint_name|null} + consumers: {module: {...}} ---
def bindings(self):
return _load(self._bd_path, {"bindings": {}, "consumers": {}})
def save_bindings(self, data):
_save(self._bd_path, data)
ui_web.py 4.7 КБ
"""outbound.ui_web — кабинет шлюза: точки, клиенты, журнал, health."""
ROUTES = [
("GET", "/api/outbound/endpoints", "endpoints", {"right": "outbound_view",
"desc": "Точки подключения (без секретов)"}),
("POST", "/api/outbound/endpoint", "endpoint", {"right": "outbound_manage",
"desc": "Точка: {action: add|update|delete, name, ...}"}),
("POST", "/api/outbound/endpoint_test", "endpoint_test", {"right": "outbound_view",
"desc": "Проверить точку {name}"}),
("GET", "/api/outbound/consumers", "consumers", {"right": "outbound_view",
"desc": "Клиенты и привязки"}),
("POST", "/api/outbound/bind", "bind", {"right": "outbound_manage",
"desc": "Привязка {module, endpoint|null}"}),
("GET", "/api/outbound/health", "health", {"right": "outbound_view",
"desc": "Сводка health точек"}),
("GET", "/api/outbound/log", "log", {"right": "outbound_view",
"desc": "Журнал вызовов (?limit)"}),
("POST", "/api/outbound/call", "call", {"right": "outbound_manage",
"desc": "Ручной вызов {consumer, op, params?} (отладка)"}),
]
SERVICE = [
{"key": "outbound", "icon": "🔌", "title": "Внешние подключения", "right": "outbound_view",
"phase": 1, "desc": "Точки, клиенты-привязки, журнал вызовов (правка — с outbound_manage)"},
]
PANELS = [
{"key": "outbound", "title": "Outbound", "visibility": "auth",
"desc": "Health точек внешних подключений"},
]
MONITOR_PANELS = [
{"id": "outbound", "title": "Outbound: точки",
"api": "/api/outbound/health", "visibility": "right:outbound_view",
"refresh_s": 30},
]
def handle_api(ctx, config, method, req):
from core.errors import UserError
if method == "endpoints":
return {"ok": True, "endpoints": ctx.api.call("outbound.endpoint_list")}
if method == "consumers":
return {"ok": True, "consumers": ctx.api.call("outbound.consumer_list")}
if method == "health":
return {"ok": True, "health": ctx.api.call("outbound.health")}
if method == "log":
query = req.get("query") or {}
return {"ok": True, "log": ctx.api.call("outbound.log",
query.get("limit") or 50)}
body = req.get("body") or {}
if method == "endpoint":
action = (body.get("action") or "").strip()
name = (body.get("name") or "").strip()
if action == "add":
return ctx.api.call("outbound.endpoint_add", name,
body.get("kind"), body.get("address"),
**{k: v for k, v in body.items()
if k not in ("action", "name", "kind",
"address")})
if action == "update":
return ctx.api.call("outbound.endpoint_update", name,
body.get("patch") or {})
if action == "delete":
return ctx.api.call("outbound.endpoint_delete", name)
raise UserError("action: add|update|delete")
if method == "endpoint_test":
return ctx.api.call("outbound.endpoint_test", body.get("name") or "")
if method == "bind":
sess = req.get("session") or {}
actor = "web:%s" % sess.get("uid") if sess.get("uid") else "?"
return ctx.api.call("outbound.bind", body.get("module") or "",
body.get("endpoint"), actor)
if method == "call":
# Отладочный ручной вызов: исполняющие и пишущие операции через
# веб запрещены (иначе outbound_manage = RCE на привязанных
# хостах + произвольная запись файлов). Модули ходят в
# outbound.call напрямую через ctx.api — их это не касается.
op = (body.get("op") or "").strip()
_blocked = ("ssh.exec", "telnet.exec", "winrm.exec", "sftp.get",
"sftp.put", "smb.get", "smb.put", "smb.mkdir",
"smb.delete", "ftp.get", "mqtt.publish", "ws.roundtrip",
"wol.send")
if op in _blocked or op.startswith(("proxmox.action",
"docker.action")):
raise UserError("op %s запрещён через веб (только чтение/статус)" % op)
return ctx.api.call("outbound.call", body.get("consumer") or "",
op, body.get("params") or {})
raise UserError("неизвестный метод: %s" % method)