Compare commits

...
42 Commits
Author SHA1 Message Date
Lyncoln 9e937d7686 Changed log level of self.address_string
Build and Publish RMManager Image / build (push) Successful in 56s
2026-10-31 09:11:39 -04:00
Lyncoln 75b4cc92da Repaired build workflow
Build and Publish RMManager Image / build (push) Successful in 46s
2026-10-30 23:41:42 -04:00
Lyncoln f90c3b1424 Added gitea workflow build
Build and Publish RMManager Image / build (push) Failing after 1m31s
2026-10-30 23:37:45 -04:00
Lyncoln 12b28f037a Updated Server to used buildListener for pings and updated the listener and firewall to handle definitions accordingly 2026-10-30 22:50:36 -04:00
Lyncoln 4ccd44937f Added dockerignore to gitignore 2026-10-30 17:28:03 -04:00
Lyncoln 53021d381e Commented out global registeredOp 2026-10-30 17:26:27 -04:00
Lyncoln 433c1e4b77 Updated response policy to use registered op and redo the request where indicated 2026-10-30 17:26:08 -04:00
Lyncoln c230939b45 Updated prints and naming conventions 2026-10-30 17:23:08 -04:00
Lyncoln 2c2a41fafb Implemented Synapse Ping as heartbeat for connection between synapse and RMASS Bridge. Also implemented set online action to update RMASS presence 2026-10-30 17:19:03 -04:00
Lyncoln b3094f75a9 Moved token refresh into its worn operation for process registry compatibility 2026-10-30 10:29:32 -04:00
Lyncoln b5591ade62 Updated resolver to use logging 2026-10-30 02:52:06 -04:00
Lyncoln fe2a54ca00 General housekeeping 2026-10-30 02:51:47 -04:00
Lyncoln f2df751294 Stopped passing outputs into executeContract and treating it as the beginning of the execution boundary 2026-10-30 02:51:11 -04:00
Lyncoln 50cb0e0924 General housekeeping 2026-10-30 02:49:29 -04:00
Lyncoln dafde0a805 Shifted to importing executeContract after filter checks to prevent excessive importing 2026-10-30 02:48:18 -04:00
Lyncoln f10ab522c9 Updated to use logging 2026-10-30 02:46:34 -04:00
Lyncoln 476ef608f3 Updated firewall to use buildThread 2026-10-30 02:18:58 -04:00
Lyncoln 63330c9ecd Imported outputs from external module to facilitate thread building and consistent runtime operation from various entrypoints 2026-10-30 02:18:24 -04:00
Lyncoln b59401c64f Migrated Monitor to create a new thread for subroutines 2026-10-30 00:44:04 -04:00
Lyncoln 4189812957 Update tracked files to respect .gitignore 2026-10-29 15:48:08 -04:00
Lyncoln 1a3caa1022 Udated runtime contract to stop doing unnecessary imports and introduced dedicated process registry. Confirmed working. 2026-10-29 15:44:45 -04:00
Lyncoln c66d139cca Stopped passing process into execution contract and action 2026-10-29 15:30:58 -04:00
Lyncoln 9b5feffe93 Added lifecycle based debugging 2026-10-29 12:15:31 -04:00
Lyncoln 1956b9bed4 Stopped returning lifecycles on lifecycleregistry 2026-10-29 02:56:11 -04:00
Lyncoln af2817cc12 Stopped returning outputs on buildRequest and executeRequest and switched to usinf executeContract in monitor 2026-10-29 02:55:28 -04:00
Lyncoln dd4abbd81a Stopped returning outputs on executeRequest 2026-10-29 02:24:17 -04:00
Lyncoln afbe0e1add Corrected commented filter for future implementation 2026-10-28 22:01:56 -04:00
Lyncoln 8fc85db44a Refined firewall by adding resolved. Added dictionary vs list handling in resolved. 2026-10-28 22:01:35 -04:00
Lyncoln a6bd89bb7e Updated apply conditions to handle more firewall rules 2026-10-28 21:23:02 -04:00
Lyncoln babd77b1ef Stopped returning outputs unnecessarily 2026-10-28 20:59:56 -04:00
Lyncoln 27d7030963 Initialized outputs outside of manageLifecycle 2026-10-28 20:58:56 -04:00
Lyncoln ad2296610c Removed old prints 2026-10-28 20:58:36 -04:00
Lyncoln d26f3a88af Removed synapse notify import 2026-10-28 20:58:19 -04:00
Lyncoln 14b5727923 Added synapse notify to preexeccution filtering at the action level. Expanded filter metadata 2026-10-28 20:57:45 -04:00
Lyncoln cdf35a9e68 Stopped returning outputs from executeContract 2026-10-28 18:12:10 -04:00
Lyncoln d5fd468498 Stopped returning outputs on buildRecursive 2026-10-28 18:04:58 -04:00
Lyncoln 24ea9a7a14 Stopped returning outputs on buildMonitor 2026-10-28 17:58:15 -04:00
Lyncoln 4b92e82ea6 Stopped returning outputs in buildOperation 2026-10-28 17:51:36 -04:00
Lyncoln 950707386a Stopped returning outputs for registerProcess 2026-10-28 17:44:09 -04:00
Lyncoln d492d78a70 Stopped returning outputs on buildAction in monitor 2026-10-28 17:37:49 -04:00
Lyncoln a574a595dc Stopped returning outputs in buildAction. Confirmed working 2026-10-28 17:37:20 -04:00
Lyncoln 717988c53b Added env variable controlled log levels 2026-10-28 17:07:55 -04:00
52 changed files with 688 additions and 493 deletions
-8
View File
@@ -1,8 +0,0 @@
.git
.gitignore
__pycache__/
*.pyc
*.pyo
*.pyd
.venv/
venv/
+182
View File
@@ -0,0 +1,182 @@
name: Build and Publish RMManager Image
on:
push:
branches: [main]
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
# ---------------------------------------------
# Checkout repository (no external actions)
# ---------------------------------------------
- name: Checkout repository
env:
GITEA_TOKEN: ${{ secrets.TEA_TOKEN }}
run: |
set -euo pipefail
BASE="${GITHUB_SERVER_URL%/}"
REPO="${GITHUB_REPOSITORY}"
git init .
git remote add origin "$BASE/$REPO.git"
if [ -n "${GITEA_TOKEN:-}" ]; then
AUTH="$(printf 'oauth2:%s' "$GITEA_TOKEN" | base64 -w0)"
git -c "http.extraHeader=Authorization: Basic $AUTH" \
fetch --depth 1 origin "$GITHUB_SHA"
else
git fetch --depth 1 origin "$GITHUB_SHA"
fi
git checkout --detach FETCH_HEAD
git config --global --add safe.directory "$PWD"
# ---------------------------------------------
# Determine image version and commit SHA
# ---------------------------------------------
- name: Determine version
id: version
run: |
set -euo pipefail
SHORT_SHA="$(git rev-parse --short=7 HEAD)"
echo "sha=$SHORT_SHA" >> "$GITHUB_OUTPUT"
if [ -f VERSION ]; then
VERSION="$(tr -d '[:space:]' < VERSION)"
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
else
echo "version=" >> "$GITHUB_OUTPUT"
fi
# ---------------------------------------------
# Login to Gitea container registry
# ---------------------------------------------
- name: Docker login
env:
REGISTRY: ${{ vars.REGISTRY }}
REGISTRY_USER: ${{ secrets.REGISTRY_USER }}
REGISTRY_TOKEN: ${{ secrets.TEA_TOKEN }}
run: |
set -euo pipefail
echo "$REGISTRY_TOKEN" | docker login "$REGISTRY" \
-u "$REGISTRY_USER" \
--password-stdin
# ---------------------------------------------
# Build and export container image
# ---------------------------------------------
- name: Build RMManager image
env:
REGISTRY: ${{ vars.REGISTRY }}
VERSION: ${{ steps.version.outputs.version }}
SHORT_SHA: ${{ steps.version.outputs.sha }}
run: |
set -euo pipefail
FULL_IMAGE="$REGISTRY/david/rmmanager"
docker build \
-f Containerfile \
-t "$FULL_IMAGE:latest" \
-t "$FULL_IMAGE:sha-$SHORT_SHA" \
--output "type=docker,dest=$RUNNER_TEMP/rmmanager-built.tar" \
.
# ---------------------------------------------
# Load exported image into Docker
# ---------------------------------------------
- name: Load RMManager image
run: |
set -euo pipefail
docker load \
--input "$RUNNER_TEMP/rmmanager-built.tar"
# ---------------------------------------------
# Add optional version tag
# ---------------------------------------------
- name: Tag RMManager image
env:
REGISTRY: ${{ vars.REGISTRY }}
VERSION: ${{ steps.version.outputs.version }}
run: |
set -euo pipefail
FULL_IMAGE="$REGISTRY/david/rmmanager"
if [ -n "$VERSION" ]; then
docker tag \
"$FULL_IMAGE:latest" \
"$FULL_IMAGE:v$VERSION"
fi
# ---------------------------------------------
# Verify local image and tags
# ---------------------------------------------
- name: Verify RMManager image
env:
REGISTRY: ${{ vars.REGISTRY }}
VERSION: ${{ steps.version.outputs.version }}
SHORT_SHA: ${{ steps.version.outputs.sha }}
run: |
set -euo pipefail
FULL_IMAGE="$REGISTRY/david/rmmanager"
docker version
docker image ls
docker image inspect "$FULL_IMAGE:latest"
docker image inspect "$FULL_IMAGE:sha-$SHORT_SHA"
if [ -n "$VERSION" ]; then
docker image inspect "$FULL_IMAGE:v$VERSION"
fi
# ---------------------------------------------
# Publish image to Gitea registry
# ---------------------------------------------
- name: Push RMManager image
env:
REGISTRY: ${{ vars.REGISTRY }}
VERSION: ${{ steps.version.outputs.version }}
SHORT_SHA: ${{ steps.version.outputs.sha }}
run: |
set -euo pipefail
FULL_IMAGE="$REGISTRY/david/rmmanager"
docker push "$FULL_IMAGE:latest"
docker push "$FULL_IMAGE:sha-$SHORT_SHA"
if [ -n "$VERSION" ]; then
docker push "$FULL_IMAGE:v$VERSION"
fi
# ---------------------------------------------
# Report successful build
# ---------------------------------------------
- name: Report build
env:
REGISTRY: ${{ vars.REGISTRY }}
VERSION: ${{ steps.version.outputs.version }}
SHORT_SHA: ${{ steps.version.outputs.sha }}
run: |
set -euo pipefail
echo "RMManager build and publish completed"
echo "Image: $REGISTRY/david/rmmanager"
echo "Latest tag: latest"
echo "Commit tag: sha-$SHORT_SHA"
if [ -n "$VERSION" ]; then
echo "Version tag: v$VERSION"
fi
+6
View File
@@ -24,3 +24,9 @@ savedata/
# Logs # Logs
*.log *.log
#Trash and unused files
trash/
#Docker files
.dockerignore
+1 -1
View File
@@ -1,5 +1,5 @@
from plugins import rmass from plugins import rmass
from core import buildOperation from core import buildOperation
outputs = buildOperation(rmass.operations.checkavailable) responses = buildOperation(rmass.operations.checkavailable)
+21 -3
View File
@@ -6,6 +6,7 @@ import logging
from .request import buildRequest from .request import buildRequest
#session = requests.Session() #session = requests.Session()
_thread_local = threading.local() _thread_local = threading.local()
@@ -25,6 +26,7 @@ def getSession(reset=False):
def executeRequest(action, outputs, resolved): def executeRequest(action, outputs, resolved):
from core.execution.lifecyclemanager.lifecyclemanager import lifecycles
session = getSession() session = getSession()
session_log = os.environ.get("LOGS_SESSION", "false").lower() == "true" session_log = os.environ.get("LOGS_SESSION", "false").lower() == "true"
@@ -36,8 +38,15 @@ def executeRequest(action, outputs, resolved):
) )
for attempt in range(2): for attempt in range(2):
request, outputs = buildRequest(action, outputs, resolved) request = buildRequest(action, outputs, resolved)
try: try:
if action.name in lifecycles:
request_info = {"Started at": time.monotonic()}
lifecycles[action.name]["Request Information"] = request_info
snapshot = lifecycles.copy()
response = session.request( response = session.request(
method=request["method"], method=request["method"],
url=request["path"], url=request["path"],
@@ -47,14 +56,23 @@ def executeRequest(action, outputs, resolved):
files=request.get("files"), files=request.get("files"),
timeout=(10, 30) timeout=(10, 30)
) )
return response, outputs
logging.debug(snapshot)
return response
except (requests.Timeout, requests.ConnectionError) as error: except (requests.Timeout, requests.ConnectionError) as error:
#print(f"{type(error).__name__} during {action.name}: {error}") #print(f"{type(error).__name__} during {action.name}: {error}")
logging.warning(f"{type(error).__name__} during {action.name}: {error}") logging.warning(f"{type(error).__name__} during {action.name}: {error}")
logging.debug("LIFECYCLE STATE BEFORE REQUEST")
logging.debug(snapshot)
logging.debug("LIFECYCLE STATE AFTER REQUEST")
logging.debug(lifecycles)
session = getSession(reset=True) session = getSession(reset=True)
if attempt == 0: if attempt == 0:
logging.warning(f"Retrying {action.name}") logging.warning(f"Retrying {action.name}")
return None, outputs return None
+1 -1
View File
@@ -46,4 +46,4 @@ def buildRequest(definition, outputs, resolved):
"image/jpeg", "image/jpeg",
), ),
} }
return request, outputs return request
+10 -9
View File
@@ -2,29 +2,30 @@ import logging
from core.apirequest import executeRequest from core.apirequest import executeRequest
from core.runtime.runtimecontract import buildRuntimeAttr, buildPreRuntimeAttr from core.runtime.runtimecontract import buildRuntimeAttr, buildPreRuntimeAttr
#from core.runtime.stopsignals import addStopSignal
def buildAction (action, outputs):
def buildAction (action, outputs, process):
action_log = True action_log = True
outputs, resolved, match, filter = buildPreRuntimeAttr(action, outputs, process) resolved, match, filter = buildPreRuntimeAttr(action, outputs)
if hasattr(action, "allowaction"): if hasattr(action, "allowaction"):
if not action.allowaction: if not action.allowaction:
return outputs, None return None
if match: if match:
return outputs, None if filter.get("execute", False):
response = executeRequest(filter["execute"], outputs, resolved)
response, outputs = executeRequest(action, outputs, resolved) return None
response = executeRequest(action, outputs, resolved)
if response is None: if response is None:
return outputs, None return outputs, None
outputs, response = buildRuntimeAttr(action, outputs, resolved, response, process) response = buildRuntimeAttr(action, outputs, resolved, response)
if action_log: if action_log:
logging.info(f"{action.name} completed") logging.info(f"{action.name} completed")
return outputs, response return response
+3
View File
@@ -1,4 +1,5 @@
import threading import threading
import logging
from .lifecyclemanager import manageLifecycle from .lifecyclemanager import manageLifecycle
@@ -12,4 +13,6 @@ def buildThread(module):
thread.start() thread.start()
logging.info(f"THREAD STARTED, {thread}")
return thread return thread
+11 -13
View File
@@ -3,31 +3,29 @@ from core.execution.operation import buildOperation
from core.execution.monitor import buildMonitor from core.execution.monitor import buildMonitor
#from core.runtime import manageFailure #from core.runtime import manageFailure
from core.execution.recursive import buildRecursive from core.execution.recursive import buildRecursive
from core.runtime.createruntime import outputs
def executeContract(module):
def executeContract(module, outputs, process):
#print("DEBUGGING")
#print(module.name)
if module.kind == "operation": if module.kind == "operation":
responses = buildOperation(module, outputs)
outputs, responses = buildOperation(module, outputs) return responses
return outputs, responses
elif module.kind == "action": elif module.kind == "action":
response = buildAction(module, outputs)
outputs, response = buildAction(module, outputs, process) return response
return outputs, response
elif module.kind == "monitor": elif module.kind == "monitor":
outputs, response= buildMonitor(module, outputs) response = buildMonitor(module, outputs)
return outputs, response return response
elif module.kind == "recursiveset": elif module.kind == "recursiveset":
outputs, response = buildRecursive(module, outputs) response = buildRecursive(module, outputs)
return outputs, response return response
#except Exception as error: #except Exception as error:
#outputs = manageFailure(error, module) #outputs = manageFailure(error, module)
@@ -6,19 +6,19 @@ from core.apirequest.execute import getSession
from core.execution.lifecyclemanager.lifecycleregistry import registerLifecycle from core.execution.lifecyclemanager.lifecycleregistry import registerLifecycle
lifecycles = {} lifecycles = {}
#outputs = {}
def manageLifecycle(module): def manageLifecycle(module):
from core.runtime.createruntime import outputs
lifecycle_log = os.environ.get("LOGS_LIFECYCLE", "false").lower() == "true" lifecycle_log = os.environ.get("LOGS_LIFECYCLE", "false").lower() == "true"
outputs = {}
registerLifecycle (module, lifecycles) registerLifecycle (module, lifecycles)
if lifecycles[module.name]["mode"] == "continuous": if lifecycles[module.name]["mode"] == "continuous":
while lifecycles[module.name]["mode"] == "continuous": while lifecycles[module.name]["mode"] == "continuous":
lifecycles[module.name]["state"] = "active" lifecycles[module.name]["state"] = "active"
outputs, response = executeContract(module, outputs, None) executeContract(module)
if lifecycle_log: if lifecycle_log:
print(f"{module.name} thread {lifecycles[module.name][module.name]} running in {lifecycles[module.name]['mode']} mode") print(f"{module.name} thread {lifecycles[module.name][module.name]} running in {lifecycles[module.name]['mode']} mode")
@@ -35,7 +35,7 @@ def manageLifecycle(module):
print(f"{module.name} thread {lifecycles[module.name][module.name]} running in {lifecycles[module.name]['mode']} mode") print(f"{module.name} thread {lifecycles[module.name][module.name]} running in {lifecycles[module.name]['mode']} mode")
print(lifecycles) print(lifecycles)
outputs, response = executeContract(module, outputs, None) response = executeContract(module)
lifecycles[module.name]["state"] = "stopped" lifecycles[module.name]["state"] = "stopped"
@@ -23,17 +23,20 @@ def registerLifecycle(module, lifecycles):
#print(lifecycles) #print(lifecycles)
if "mode" not in lifecycles[module.name]: if "mode" not in lifecycles[module.name]:
mode = getattr(module, "mode") if hasattr(module, "mode"):
lifecycles[module.name]["mode"] = mode mode = getattr(module, "mode")
lifecycles[module.name]["mode"] = mode
else:
lifecycles[module.name]["mode"] = "oneshot"
if lifecycles[module.name]["mode"] == "continuous": if lifecycles[module.name]["mode"] == "continuous":
lifecycles[module.name]["refresh_rate"] = module.sleeptime lifecycles[module.name]["refresh_rate"] = module.sleeptime
if not lifecycles[module.name]["mode"]: #if lifecycles[module.name]["mode"] == {}:
lifecycles[module.name]["mode"] = "oneshot" # lifecycles[module.name]["mode"] = "oneshot"
#lifecycles[module.name][registryname] = registryvalue #lifecycles[module.name][registryname] = registryvalue
return lifecycles return
+16 -14
View File
@@ -1,12 +1,8 @@
import logging import logging
#from core.execution.executioncontract import executeContract
from core.execution.action import buildAction
from core.execution.operation import buildOperation
from core.apirequest import executeRequest from core.apirequest import executeRequest
from core.runtime.resolver import resolveItem from core.runtime.resolver import resolveItem
from core.runtime.applyconditions import applyConditions from core.runtime.applyconditions import applyConditions
#from plugins.rmass.actions import login, refresh_tokens
from core.runtime import buildOutput from core.runtime import buildOutput
from core.runtime.savedata import loadSave from core.runtime.savedata import loadSave
@@ -14,12 +10,17 @@ outputs = {}
resolved = {} resolved = {}
def buildMonitor(monitor, outputs): def buildMonitor(monitor, outputs):
outputs = loadSave(monitor, outputs) loadSave(monitor, outputs)
response, outputs = executeRequest(monitor, outputs, resolved) response = executeRequest(monitor, outputs, resolved)
##Potentially replace response processing with responsePolicy
if response is None: if response is None:
return outputs, None return None
if not response.ok:
logging.warning(f"Response with status code {response.status_code} during {monitor.name}")
return response
responsedata = response.json() responsedata = response.json()
@@ -30,13 +31,14 @@ def buildMonitor(monitor, outputs):
buildOutput(response, monitor, outputs) buildOutput(response, monitor, outputs)
if match: if match:
from core.execution.createthread import buildThread
#from core.execution.executioncontract import executeContract
logging.info(filter.get("print")) logging.info(filter.get("print"))
module = filter["execute"] #module = filter["execute"]
if module.kind == "action": #response = executeContract(filter["execute"], outputs)
outputs, response = buildAction(module, outputs, None)
if module.kind == "operation":
outputs, responses = buildOperation(module, outputs)
response = responses
return outputs, response thread = buildThread(filter["execute"])
return response
+23 -29
View File
@@ -5,75 +5,69 @@ from rich.console import Console
from core.runtime.managefailure import manageFailure from core.runtime.managefailure import manageFailure
from core.runtime.stopsignals import checkDependency from core.runtime.stopsignals import checkDependency
from core.execution.processregistry import registerProcess from core.execution import processregistry
from core.console import console from core.console import console
outputs = {} #outputs = {}
#console = Console(stderr=True)
def buildOperation(definition, outputs): def buildOperation(definition, outputs):
process_log = True process_log = True
from core.execution.executioncontract import executeContract
actions = definition.actions actions = definition.actions
outputs = registerProcess(definition, outputs)
responses = [] responses = []
currentOp = processregistry.registerProcess(definition)
if getattr(definition, "track", False): if getattr(definition, "track", False):
actions = track( actions = track(
actions, actions,
#description=definition.name
description=f"[green]{definition.name}[/green]", description=f"[green]{definition.name}[/green]",
console=console console=console
) )
for action in actions: for action in actions:
processregistry.registeredOp = currentOp
#Check if dependencies are in stop signal registry stop = checkDependency(action)
stop = checkDependency(action, outputs, definition)
if stop: if stop:
continue continue
#Register action before its execution
if action.kind == "action": if action.kind == "action":
outputs["processes"][definition.name][action.name] = {} currentOp[action.name] = {}
#Pass the definition through to the action so it can be passed into buildPreRuntimeAttr from core.execution.executioncontract import executeContract
#Definition should be passed through in order to give access to addStopSignal so it can write directly to the definition during buildPreRuntimeAttr
#Stop signal can optionally be engaged by doing addStopSignal inside the action definition response = executeContract(action)
outputs, response = executeContract(action, outputs, definition)
responses.append(response) responses.append(response)
#Register responses in process registry
if action.kind == "action": if action.kind == "action":
outputs["processes"][definition.name][action.name]["response"] = response currentOp[action.name]["response"] = response
##Append completed process to process registry if action.alias in currentOp["stopsignals"]:
outputs["processes"][definition.name]["completed"].append(action.alias) continue
currentOp["completed"].append(action.alias)
#Temporarily disabled manageFailure functions #Temporarily disabled manageFailure functions
#except Exception as error: #except Exception as error:
#outputs = manageFailure(error, action, outputs) #outputs = manageFailure(error, action, outputs)
#Print completion status after runtime processes
if process_log: if process_log:
stopsignal_count = len(outputs["processes"][definition.name]["stopsignals"]) completed_count = len(currentOp["completed"])
completed_count = len(outputs["processes"][definition.name]["completed"])
if completed_count == 1 and stopsignal_count == 1: if completed_count == 0:
logging.info(f"{definition.name} bypassed") logging.info(f"{definition.name} bypassed")
else: else:
logging.info(f"{definition.name} completed") logging.info(f"{definition.name} completed")
#For debugging purposes only
process_registry_logs = False process_registry_logs = False
if process_registry_logs: if process_registry_logs:
pprint.pprint(outputs["processes"]) import pprint
pprint.pprint(processregistry.processes)
return responses
return outputs, responses
+11 -9
View File
@@ -1,12 +1,14 @@
def registerProcess(module, outputs): processes = {}
if not "processes" in outputs: registeredOp = {}
outputs["processes"] = {}
if not module.name in outputs["processes"]: def registerProcess(module):
outputs["processes"][module.name] = {} #global registeredOp
#outputs["processes"][module.name] = {} processes[module.name] = {}
outputs["processes"][module.name]["stopsignals"] = []
outputs["processes"][module.name]["completed"] = []
return outputs registeredOp = processes[module.name]
processes[module.name]["stopsignals"] = []
processes[module.name]["completed"] = []
return registeredOp
+22 -26
View File
@@ -2,7 +2,7 @@ import logging
import os import os
from core.runtime.stopsignals import checkDependency from core.runtime.stopsignals import checkDependency
from core.execution.processregistry import registerProcess from core.execution import processregistry
from rich.progress import track from rich.progress import track
from core.console import console from core.console import console
@@ -10,8 +10,6 @@ def buildRecursive(definition, outputs):
process_log = True process_log = True
recursive_log = os.environ.get("LOGS_RECURSIVE", "false").lower() == "true" recursive_log = os.environ.get("LOGS_RECURSIVE", "false").lower() == "true"
#print(outputs[definition.recursiveset["source"]])
source = outputs.get(definition.recursiveset["source"]) source = outputs.get(definition.recursiveset["source"])
if source is None or source == []: if source is None or source == []:
@@ -19,10 +17,10 @@ def buildRecursive(definition, outputs):
logging.debug(f"No items available in {source} for {definition.name}") logging.debug(f"No items available in {source} for {definition.name}")
return outputs, [] return outputs, []
outputs = registerProcess(definition, outputs) currentOp = processregistry.registerProcess(definition)
responses = [] responses = []
from core.execution.executioncontract import executeContract
recursiveset = definition.recursiveset["actions"] recursiveset = definition.recursiveset["actions"]
source = definition.recursiveset["source"] source = definition.recursiveset["source"]
@@ -32,7 +30,6 @@ def buildRecursive(definition, outputs):
reverse = definition.recursiveset.get("reverse", False) reverse = definition.recursiveset.get("reverse", False)
if reverse: if reverse:
#print("reversing")
items = reversed(items) items = reversed(items)
for item in track( for item in track(
@@ -46,37 +43,30 @@ def buildRecursive(definition, outputs):
outputs["stopsignals"] = [] outputs["stopsignals"] = []
for job in recursiveset: for job in recursiveset:
processregistry.registeredOp = currentOp
stop = checkDependency(job, outputs, definition) stop = checkDependency(job)
#print(f"STOP SIGNAL DEBUGGING during {job.name}")
#print(outputs["stopsignals"])
#print(stop)
if stop: if stop:
#outputs["processes"][definition.name]["stopsignals"].append(job.alias)
#print("STOP SIGNAL DETECTED")
#print(f"Stopping {job.name} due to stop signal")
continue continue
#print(f"RECURSIVE DEBUG DURING {job.name}")
#print(item)
#outputs, response = buildAction(job, outputs)
if job.kind == "action": if job.kind == "action":
outputs["processes"][definition.name][job.name] = {} currentOp[job.name] = {}
outputs, response = executeContract(job, outputs, definition) from core.execution.executioncontract import executeContract
response = executeContract(job)
responses.append(response) responses.append(response)
if job.kind == "action":
outputs["processes"][definition.name][job.name]["response"] = response
outputs["processes"][definition.name]["completed"].append(job.alias) if job.kind == "action":
currentOp[job.name]["response"] = response
currentOp["completed"].append(job.alias)
if process_log: if process_log:
stopsignal_count = len(outputs["processes"][definition.name]["stopsignals"]) completed_count = len(currentOp["completed"])
completed_count = len(outputs["processes"][definition.name]["completed"])
if completed_count == 1 and stopsignal_count == 1: if completed_count == 0:
logging.info(f"{definition.name} bypassed") logging.info(f"{definition.name} bypassed")
else: else:
logging.info(f"{definition.name} completed") logging.info(f"{definition.name} completed")
@@ -88,4 +78,10 @@ def buildRecursive(definition, outputs):
outputs.pop("stopsignals", None) outputs.pop("stopsignals", None)
return outputs, responses process_registry_logs = False
if process_registry_logs:
import pprint
pprint.pprint(processregistry.processes)
return responses
+12 -3
View File
@@ -17,7 +17,12 @@ def applyConditions(definition, dictionary, outputs):
if filtervalue is None: if filtervalue is None:
continue continue
resolvedvalue = dictionary.get(filter["filterkey"]) if filter.get("filterkey_format", None):
if filter["filterkey_format"] == "length":
resolvedvalue = len(dictionary.get(filter["filterkey"]))
else:
resolvedvalue = dictionary.get(filter["filterkey"])
if filter["operator"] == ">=": if filter["operator"] == ">=":
if int(resolvedvalue or 0) >= int(filtervalue): if int(resolvedvalue or 0) >= int(filtervalue):
@@ -44,8 +49,12 @@ def applyConditions(definition, dictionary, outputs):
return True, filter return True, filter
if filter["operator"] == "in": if filter["operator"] == "in":
if filtervalue in dictionary: if filter["filterkey"] == "called_dictionary":
return True, filter if filtervalue in dictionary:
return True, filter
else:
if filtervalue in dictionary.get(filter["filterkey"], {}):
return True, filter
if filter["operator"] == "is": if filter["operator"] == "is":
if resolvedvalue is filtervalue: if resolvedvalue is filtervalue:
+4
View File
@@ -0,0 +1,4 @@
outputs = {}
def getOuputs():
return outputs
+25 -24
View File
@@ -2,37 +2,38 @@ import os
import logging import logging
from core.runtime.applyconditions import applyConditions from core.runtime.applyconditions import applyConditions
from core.runtime.resolver import resolveItem
def buildFirewall(listener, outputs): def buildFirewall(listener, outputs):
#firewall_log = os.environ.get("LOGS_FIREWALL", "false").lower() == "true" if hasattr(listener, "filters"):
firewall_log = True #firewall_log = os.environ.get("LOGS_FIREWALL", "false").lower() == "true"
firewall_log = True
status_code = 200 status_code = 200
outputs["firewall_allowed"] = [] outputs["firewall_allowed"] = []
for item in outputs[listener.recursive["source"]]: for item in outputs[listener.recursive["source"]]:
resolved = resolveItem(listener, item)
match, filter = applyConditions(listener, item, outputs) match, filter = applyConditions(listener, resolved, outputs)
if match: if match:
status_code = filter.get("status_code", 200) status_code = filter.get("status_code", 200)
if firewall_log:
logging.info(filter.get("print"))
logging.info(f"Returning status_code: {status_code}")
continue
#This needs to be moved into definitions but until logic is put in place it lives here. if filter.get("execute", False):
if "com.loxthelion.rmass_message_id" in item.get("content", {}): from core.execution.createthread import buildThread
status_code = 200
if firewall_log:
logging.info("Excluding previously imported items")
logging.info(f"Returning status_code: {status_code}")
continue thread = buildThread(filter["execute"])
outputs["firewall_allowed"].append(item) if firewall_log:
logging.info(filter.get("print"))
logging.info(f"Returning status_code: {status_code}")
continue
if outputs["firewall_allowed"] != []: outputs["firewall_allowed"].append(item)
return True, status_code
#Needs to be present to prevent TypeError if outputs["firewall_allowed"] != []:
return False, status_code return True, status_code
#Needs to be present to prevent TypeError
return False, status_code
+1 -1
View File
@@ -22,7 +22,7 @@ def manageFailure(error, action, outputs):
else: else:
print(f"{type(error).__name__} detected during {action.name}. Applying recovery policy.") print(f"{type(error).__name__} detected during {action.name}. Applying recovery policy.")
outputs, response = buildAction(repair, outputs, None) response = buildAction(repair, outputs)
return outputs return outputs
print(f"{error}") print(f"{error}")
+8 -3
View File
@@ -1,7 +1,9 @@
import os import os
import logging
def resolveItem(action, item): def resolveItem(action, item):
resolver_log = os.environ.get("LOGS_RESOLVER", "false").lower() == "true" #resolver_log = os.environ.get("LOGS_RESOLVER", "false").lower() == "true"
resolver_log = True
resolved = {} resolved = {}
if hasattr(action, "resolve"): if hasattr(action, "resolve"):
@@ -9,11 +11,14 @@ def resolveItem(action, item):
target = item target = item
try: try:
for key in keys: for key in keys:
target = target[key] if isinstance(target, dict):
target = target.get(key)
else:
target = target[key]
except (KeyError, IndexError) as error: except (KeyError, IndexError) as error:
if resolver_log: if resolver_log:
print(f"Resolver target key/index {error} not found during {action.name}") logging.info(f"Resolver target key/index {error} not found during {action.name}")
target = None target = None
resolved[label] = target resolved[label] = target
+16 -2
View File
@@ -1,9 +1,15 @@
import logging import logging
from core.apirequest import executeRequest from core.apirequest import executeRequest
from core.execution import processregistry
def responsePolicy(response, action, outputs, resolved): def responsePolicy(response, action, outputs, resolved):
if hasattr(action, "responsepolicy"): if hasattr(action, "responsepolicy"):
#if processregistry.registeredOp:
#currentOp = processregistry.registeredOp
#print(registeredOp[action.name]["response"])
#print(currentOp)
if not response.ok: if not response.ok:
logging.warning(f"{response} during {action.name}") logging.warning(f"{response} during {action.name}")
@@ -11,11 +17,19 @@ def responsePolicy(response, action, outputs, resolved):
if policy["statuscode"] == response.status_code: if policy["statuscode"] == response.status_code:
logging.warning(policy["print"]) logging.warning(policy["print"])
newresponse, outputs = executeRequest(policy["execute"], outputs, resolved) from core.execution.executioncontract import executeContract
executeContract(policy["execute"])
#newresponse = executeRequest(action, outputs, resolved)
if policy["response_update"]: if policy["response_update"]:
logging.info("Updating response") logging.info("Updating response")
print("Updating response") newresponse = executeRequest(action, outputs, resolved)
#response = newresponse
if processregistry.registeredOp:
response = processregistry.registeredOp[action.name]["response"]
else:
response = newresponse response = newresponse
break break
+8 -8
View File
@@ -2,27 +2,27 @@ import logging
from core.runtime.stopsignals import addStopSignal from core.runtime.stopsignals import addStopSignal
def buildPreRuntimeAttr(action, outputs, process): def buildPreRuntimeAttr(action, outputs):
from core.runtime import loadSave, readRow from core.runtime import loadSave, readRow
from core.runtime.applyconditions import applyConditions from core.runtime.applyconditions import applyConditions
from core.runtime.resolver import resolveItem from core.runtime.resolver import resolveItem
outputs = loadSave(action, outputs) loadSave(action, outputs)
resolved = resolveItem(action, outputs) resolved = resolveItem(action, outputs)
resolved, outputs = readRow(action, resolved, outputs) readRow(action, resolved, outputs)
match, filter = applyConditions(action, resolved, outputs) match, filter = applyConditions(action, resolved, outputs)
if match: if match:
logging.info(filter.get("print")) logging.info(filter.get("print"))
addStopSignal(action, outputs, process) addStopSignal(action)
return outputs, resolved, match, filter return resolved, match, filter
def buildRuntimeAttr(action, outputs, resolved, response, process): def buildRuntimeAttr(action, outputs, resolved, response):
from core.runtime.resolver import extractItem from core.runtime.resolver import extractItem
from core.runtime.responsepolicy import responsePolicy from core.runtime.responsepolicy import responsePolicy
from core.runtime.image import buildImage from core.runtime.image import buildImage
@@ -33,7 +33,7 @@ def buildRuntimeAttr(action, outputs, resolved, response, process):
buildOutput(response, action, outputs) buildOutput(response, action, outputs)
if not response.ok: if not response.ok:
addStopSignal(action, outputs, process) addStopSignal(action)
return outputs, response return outputs, response
extractItem(action, outputs, resolved) extractItem(action, outputs, resolved)
@@ -48,4 +48,4 @@ def buildRuntimeAttr(action, outputs, resolved, response, process):
buildLog(action, outputs, response) buildLog(action, outputs, response)
return outputs, response return response
+6 -6
View File
@@ -41,7 +41,7 @@ def buildSave(action, outputs):
if savedata_log: if savedata_log:
logging.info(f"Data saved by {action.name} to {action.savepath}") logging.info(f"Data saved by {action.name} to {action.savepath}")
return outputs return
def loadSave(action, outputs): def loadSave(action, outputs):
savedata_log = os.environ.get("LOGS_SAVE_DATA_ACCESS", "false").lower() == "true" savedata_log = os.environ.get("LOGS_SAVE_DATA_ACCESS", "false").lower() == "true"
@@ -55,7 +55,7 @@ def loadSave(action, outputs):
if savedata_log: if savedata_log:
logging.info(f"Data loaded by {action.name} from {action.savepath}") logging.info(f"Data loaded by {action.name} from {action.savepath}")
return outputs return
def saveRow(action, outputs, resolved): def saveRow(action, outputs, resolved):
import sqlite3 import sqlite3
@@ -151,7 +151,7 @@ def saveRow(action, outputs, resolved):
logging.info(f"Successfully saved data to {table} during {action.name}") logging.info(f"Successfully saved data to {table} during {action.name}")
return True return
def readRow(action, resolved, outputs): def readRow(action, resolved, outputs):
@@ -221,9 +221,9 @@ def readRow(action, resolved, outputs):
logging.debug(outputs[output_name]) logging.debug(outputs[output_name])
logging.debug(dbtarget) logging.debug(dbtarget)
return resolved, outputs return
else: else:
logging.info(f"Search target {value} not found in database table {table}") logging.info(f"Search target {value} not found in database table {table}")
return resolved, outputs return
return resolved, outputs return
+10 -8
View File
@@ -1,9 +1,10 @@
#def checkDependency(action, outputs): #from core.execution.processregistry import registeredOp
def checkDependency(action, outputs, operation):
def checkDependency(action):
from core.execution.processregistry import registeredOp
if hasattr(action, "depends"): if hasattr(action, "depends"):
stop = any( stop = any(
dependency in outputs["processes"][operation.name]["stopsignals"] dependency in registeredOp["stopsignals"]
#dependency in outputs["stopsignals"]
for dependency in action.depends for dependency in action.depends
) )
@@ -12,9 +13,10 @@ def checkDependency(action, outputs, operation):
return False return False
#def addStopSignal(action, outputs): def addStopSignal(action):
def addStopSignal(action, outputs, operation): from core.execution.processregistry import registeredOp
if hasattr(action, "depends"): if hasattr(action, "depends"):
outputs["processes"][operation.name]["stopsignals"].append(action.alias)
#outputs["stopsignals"].append(action.alias) registeredOp["stopsignals"].append(action.alias)
return return
+30 -23
View File
@@ -2,43 +2,50 @@ import json
import logging import logging
from core.runtime.outputs import buildOutput from core.runtime.outputs import buildOutput
from core.execution.executioncontract import executeContract
from core.runtime.firewall import buildFirewall from core.runtime.firewall import buildFirewall
from core.runtime.createruntime import outputs
from core.execution.processregistry import processes
def buildListener(listener, self): def buildListener(listener, self):
if self.path == "/_matrix/app/v1/ping":
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.end_headers()
self.wfile.write(b"{}")
return
if self.path.startswith(listener.acceptpath): if self.path.startswith(listener.acceptpath):
if hasattr(listener, "print"):
#Prints can used buildlog for listener runtime logging.info(listener.print)
logging.info("Matrix transaction received")
content_length = int(self.headers["Content-Length"]) content_length = int(self.headers["Content-Length"])
data = self.rfile.read(content_length) bytes = self.rfile.read(content_length)
request = json.loads(data)
outputs = {} #request = json.loads(data)
buildOutput(data, listener, outputs) buildOutput(bytes, listener, outputs)
allow, status_code = buildFirewall(listener, outputs) if hasattr(listener, "filters"):
allow, status_code = buildFirewall(listener, outputs)
if allow: if allow:
#Print allowed items for logging purposes logging.info(outputs["firewall_allowed"])
logging.info(outputs["firewall_allowed"])
#Process and forward to RMASS if hasattr(listener, "execute"):
outputs, responses = executeContract(listener.execute, outputs, None) if listener.execute["engine"] == "thread":
from core.execution.createthread import buildThread
buildThread(listener.execute["module"])
elif listener.execute["engine"] == "contract":
from core.execution.executioncontract import executeContract
executeContract(listener.execute["module"])
else:
if hasattr(listener, "execute"):
if listener.execute["engine"] == "thread":
from core.execution.createthread import buildThread
buildThread(listener.execute["module"])
elif listener.execute["engine"] == "contract":
from core.execution.executioncontract import executeContract
executeContract(listener.execute["module"])
status_code = listener.status_code["success"]
self.send_response(status_code) self.send_response(status_code)
self.send_header("Content-Type", "application/json") self.send_header("Content-Type", "application/json")
@@ -49,4 +56,4 @@ def buildListener(listener, self):
else: else:
self.send_error(404) self.send_error(404)
return outputs return
+92 -70
View File
@@ -4,106 +4,128 @@ import logging
from http.server import HTTPServer, BaseHTTPRequestHandler from http.server import HTTPServer, BaseHTTPRequestHandler
from .dispatcher import dispatchRequest
from .createresponse import buildResponse #from .dispatcher import dispatchRequest
from plugins.rmass.listeners import synapselistener #from .createresponse import buildResponse
from plugins.rmass.listeners import synapselistener, matrixpinglistener
from core.server.createlistener import buildListener from core.server.createlistener import buildListener
#from plugins.rmass.actions import pingsynapse #from plugins.rmass.actions import synapsesetbridgeonline
port = int(os.environ.get("SERVER_PORT", 19001)) port = int(os.environ.get("SERVER_PORT", 19001))
class RequestHandler(BaseHTTPRequestHandler): class RequestHandler(BaseHTTPRequestHandler):
def log_message(self, format, *args):
logging.info(
"%s - %s",
self.address_string(),
format % args
)
def do_POST(self): def do_POST(self):
#lifecycles = {} logging.debug("POST request received")
print("POST request received")
max_request_size = 1024 buildListener(matrixpinglistener, self)
content_length = self.headers.get("Content-Length") # if self.path == "/_matrix/app/v1/ping":
# Check whether Content-Length exists # from core.execution.createthread import buildThread
if content_length is None: # thread = buildThread(synapsesetbridgeonline)
response = {
"status": "error",
"message": "Content length required"
}
status = 411
else: # self.send_response(200)
# Attempt to convert the header into an integer # self.send_header("Content-Type", "application/json")
try: # self.send_header("Content-Length", "2")
length = int(content_length) # self.end_headers()
except ValueError: # self.wfile.write(b"{}")
length = None # return
# Validate the converted length # else:
if length is None: # print("POST request received")
response = {
"status": "error",
"message": "Invalid Content-Length"
}
status = 400
elif length < 0: # max_request_size = 1024
response = {
"status": "error",
"message": "Bad Request"
}
status = 400
elif length > max_request_size: # content_length = self.headers.get("Content-Length")
response = {
"status": "error",
"message": "Content exceeds maximum request size"
}
status = 413
else: # # Check whether Content-Length exists
# Content-Length is valid, so read the body # if content_length is None:
data = self.rfile.read(length) # response = {
# "status": "error",
# "message": "Content length required"
# }
# status = 411
try: # else:
request = json.loads(data) # # Attempt to convert the header into an integer
# try:
# length = int(content_length)
# except ValueError:
# length = None
except json.JSONDecodeError: # # Validate the converted length
response = { # if length is None:
"status": "error", # response = {
"message": "Invalid JSON received" # "status": "error",
} # "message": "Invalid Content-Length"
status = 400 # }
# status = 400
else: # elif length < 0:
# Only valid JSON reaches the dispatcher # response = {
thread, lifecycles, module, mode = dispatchRequest( # "status": "error",
request, lifecycles # "message": "Bad Request"
) # }
# status = 400
response = buildResponse(module, thread, mode) # elif length > max_request_size:
status = 200 # response = {
# "status": "error",
# "message": "Content exceeds maximum request size"
# }
# status = 413
# Serialize the response # else:
responsebytes = json.dumps(response).encode("utf-8") # # Content-Length is valid, so read the body
# data = self.rfile.read(length)
# Send one HTTP response # try:
self.send_response(status) # request = json.loads(data)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(responsebytes)))
self.end_headers()
self.wfile.write(responsebytes) # except json.JSONDecodeError:
# response = {
# "status": "error",
# "message": "Invalid JSON received"
# }
# status = 400
# else:
# # Only valid JSON reaches the dispatcher
# thread, lifecycles, module, mode = dispatchRequest(
# request, lifecycles
# )
# response = buildResponse(module, thread, mode)
# status = 200
# # Serialize the response
# responsebytes = json.dumps(response).encode("utf-8")
# # Send one HTTP response
# self.send_response(status)
# self.send_header("Content-Type", "application/json")
# self.send_header("Content-Length", str(len(responsebytes)))
# self.end_headers()
# self.wfile.write(responsebytes)
def do_PUT(self): def do_PUT(self):
#print("PUT request received") logging.debug("PUT request received")
outputs = buildListener(synapselistener, self) buildListener(synapselistener, self)
def startServer(): def startServer():
server = HTTPServer(("0.0.0.0", port), RequestHandler) server = HTTPServer(("0.0.0.0", port), RequestHandler)
logging.info(f"Server listening on port {port}") logging.info(f"Server listening on port {port}")
#outputs = {}
#outputs, response = buildAction(pingsynapse, outputs)
server.serve_forever() server.serve_forever()
+17 -11
View File
@@ -1,17 +1,18 @@
import threading import threading
import logging import logging
import os
from core.server.server import startServer from core.server.server import startServer
#from core.execution.lifecyclemanager.lifecycleregistry import registerLifecycle
from core.execution.createthread import buildThread from core.execution.createthread import buildThread
from plugins.rmass.monitors import rmassmonitor from plugins.rmass.monitors import rmassmonitor
from plugins.rmass.actions import refresh_tokens from plugins.rmass.operations import token_update
from plugins.rmass.actions import pingsynapse
from core.console import console from core.console import console
from rich.logging import RichHandler from rich.logging import RichHandler
logging.basicConfig( logging.basicConfig(
level=logging.WARNING, level=os.environ.get("LOG_LEVEL", "WARNING").upper(),
format="%(message)s", format="%(message)s",
handlers=[ handlers=[
RichHandler( RichHandler(
@@ -27,16 +28,21 @@ logging.basicConfig(
import login import login
import synapsemessagepush import synapsemessagepush
rmassmonitor.mode = "continuous"
refresh_tokens.mode = "continuous"
thread = buildThread(rmassmonitor)
thread = buildThread(refresh_tokens)
thread = threading.Thread( thread = threading.Thread(
target=startServer, target=startServer,
#args=(module, lifecycles)
name="RMASS Bridge Server" name="RMASS Bridge Server"
) )
thread.start() thread.start()
pingsynapse.mode = "continuous"
rmassmonitor.mode = "continuous"
token_update.mode = "continuous"
#refresh_tokens.mode = "continuous"
thread = buildThread(rmassmonitor)
#thread = buildThread(refresh_tokens)
thread = buildThread(token_update)
thread = buildThread(pingsynapse)
+1 -1
View File
@@ -3,5 +3,5 @@ from core.execution.operation import buildOperation
outputs = {} outputs = {}
outputs = buildOperation(rmass.operations.login, outputs) responses = buildOperation(rmass.operations.login, outputs)
+1 -1
View File
@@ -7,6 +7,6 @@ definition = rmassmonitor
outputs = {} outputs = {}
resolved = {} resolved = {}
while True: while True:
outputs = buildMonitor(definition, outputs) response = buildMonitor(definition, outputs)
time.sleep(definition.refreshrate) time.sleep(definition.refreshrate)
+1 -1
View File
@@ -2,5 +2,5 @@ from plugins import rmass
from core import buildAction from core import buildAction
outputs = {} outputs = {}
outputs = buildAction(rmass.actions.pingsynapse, outputs) response = buildAction(rmass.actions.pingsynapse, outputs)
+4 -2
View File
@@ -1,11 +1,13 @@
name = "Synapse Wake" name = "Synapse Ping"
kind = "action" kind = "action"
alias = "synapseping" alias = "pingsynapse"
showstatus = True showstatus = True
sleeptime = 15
isrequest = True isrequest = True
method = "POST" method = "POST"
+11
View File
@@ -5,6 +5,8 @@ name = "Token refresh"
kind = "action" kind = "action"
alias = "refresh_tokens"
showstatus = True showstatus = True
method = "GET" method = "GET"
@@ -44,6 +46,15 @@ outputs = {
"refresh_token": "refreshToken", "refresh_token": "refreshToken",
} }
responsepolicy = [
{
"statuscode": 403,
"print": "Refreshing stale tokens...",
"execute": login,
"response_update": False,
}
]
recoverPolicy = { recoverPolicy = {
FileNotFoundError: [ FileNotFoundError: [
getcsrf, getcsrf,
@@ -88,6 +88,15 @@ outputs = {
##Filter test against recursiveset ##Filter test against recursiveset
filters = [ filters = [
{
"filterkey": "message_body",
#"filterkey_lib": "resolved",
"filterkey_format": "length",
"execute": synapsenotify,
"operator": "<",
"filtervalue": 5,
"print": f"Message body is less than 5 characters. Notifying with {synapsenotify.name}...",
},
{ {
"filterkey": "msgtype", "filterkey": "msgtype",
"operator": "==", "operator": "==",
+2 -9
View File
@@ -10,15 +10,8 @@ showstatus = True
resolve = { resolve = {
"get": { "get": {
"sender": ["room_name"], "event_id": ["events", 0, "event_id"],
"event_id": ["outmessageitem", "event_id"], "room_id": ["events", 0, "room_id"],
"room_id": ["outmessageitem", "room_id"],
},
"deconstruct": {
"sender" : [
"#rmass_dm_",
":loxthelion.com",
],
}, },
} }
@@ -1,5 +1,3 @@
from . import synapsenotify
name = "Synapse Message Push" name = "Synapse Message Push"
kind = "action" kind = "action"
@@ -0,0 +1,36 @@
name = "Set RMASS Bridge Online"
kind = "action"
alias = "setrmassbridgeonline"
showstatus = True
#sleeptime = 15
isrequest = True
method = "PUT"
path = [
"https://loxthelion.com/_matrix/client/v3/presence/@rmass_bridge:loxthelion.com/status",
]
headers = {
"Authorization": {
"os": "AS_TOKEN",
"prefix": "Bearer ",
},
"Content-Type": "application/json",
}
keys = [
{
"keytitle": "presence",
"keyvalue": "online",
},
{
"keytitle": "status_msg",
"keyvalue": "RMASS Bridge Connected",
},
]
@@ -0,0 +1,16 @@
from plugins.rmass.actions import synapsesetbridgeonline
name = "Matrix Ping Listener"
kind = "listener"
acceptpath = "/_matrix/app/v1/ping"
execute = {
"module": synapsesetbridgeonline,
"engine": "thread",
}
status_code = {
"success": 200
}
+37 -2
View File
@@ -1,4 +1,5 @@
from plugins.rmass.operations import rmassmessagepush from plugins.rmass.operations import rmassmessagepush
from plugins.rmass.actions import synapsenotify
name = "Synapse Listener" name = "Synapse Listener"
@@ -8,6 +9,19 @@ outputs = {
"events": "events", "events": "events",
} }
resolve = {
"get": {
"sender": ["sender"],
"event_id": ["event_id"],
"message_body": ["content", "body"],
"content": ["content"],
"type": ["type"],
"room_id": ["room_id"],
"timestamp": ["origin_server_ts"],
"filter_item": []
}
}
recursive = { recursive = {
"source": "events", "source": "events",
"reverse": False, "reverse": False,
@@ -16,11 +30,14 @@ recursive = {
acceptpath = "/_matrix/app/v1/transactions/" acceptpath = "/_matrix/app/v1/transactions/"
execute = rmassmessagepush execute = {
"module": rmassmessagepush,
"engine": "contract",
}
filters = [ filters = [
{ {
"filterkey": "event", "filterkey": "filter_item",
"operator": "in", "operator": "in",
"filtervalue": "state_key", "filtervalue": "state_key",
"print": "Excluding state.key event", "print": "Excluding state.key event",
@@ -40,6 +57,24 @@ filters = [
"print": "Excluding non-room message event", "print": "Excluding non-room message event",
"status_code": 200, "status_code": 200,
}, },
{
"filterkey": "content",
"operator": "in",
"filtervalue": "com.loxthelion.rmass_message_id",
"print": "Excluding imported message loop item",
"status_code": 200,
},
{
"filterkey": "message_body",
#"filterkey_lib": "resolved",
"filterkey_format": "length",
"execute": synapsenotify,
"operator": "<",
"filtervalue": 5,
"print": f"Message body is less than 5 characters. Notifying with {synapsenotify.name}...",
"status_code":200
},
] ]
print = "Matrix transaction received"
+1 -1
View File
@@ -1,6 +1,6 @@
from plugins.rmass.actions import getcsrf, login from plugins.rmass.actions import getcsrf, login
name = "Login" name = "Login Operation"
kind = "operation" kind = "operation"
+13
View File
@@ -0,0 +1,13 @@
import os
from plugins.rmass.actions import refresh_tokens
name = "Token Refresh Operation"
kind = "operation"
actions = [
refresh_tokens,
]
sleeptime = os.environ.get("TOKEN_REFRESH_RATE", 105)
+1 -1
View File
@@ -5,6 +5,6 @@ from core.execution.operation import buildOperation
outputs ={} outputs ={}
while True: while True:
outputs = buildOperation(rmass.operations.setavailable, outputs) responses = buildOperation(rmass.operations.setavailable, outputs)
time.sleep(rmass.operations.setavailable.sleeptime) time.sleep(rmass.operations.setavailable.sleeptime)
+1 -1
View File
@@ -3,5 +3,5 @@ from core.execution.operation import buildOperation
outputs = {} outputs = {}
outputs = buildOperation(rmass.operations.synapsemessagepush, outputs) responses = buildOperation(rmass.operations.synapsemessagepush, outputs)
-5
View File
@@ -1,5 +0,0 @@
from plugins import rmass
from core.operation.operation import buildOperation
outputs = buildOperation(rmass.checkavailableoperation)
-17
View File
@@ -1,17 +0,0 @@
import importlib
from lifecyclemanager import manageLifecycle
from dispatchstate import buildState
def buildDispatch(request):
dispatch = []
for key in request["module"]:
dispatch.append(request["module"][key])
resolveddispatch = "plugins."+ ".".join(dispatch)
module = importlib.import_module(resolveddispatch)
mode = request.get("mode", "oneshot")
return module, mode
-30
View File
@@ -1,30 +0,0 @@
import threading
from lifecycleregistry import registerLifecycle
from lifecyclemanager import manageLifecycle
threadlist = []
threadnum = 0
def buildThread(module, lifecycles):
global threadnum
threadnum += 1
threadname = f"thread.{threadnum}"
thread = threading.Thread(
target=manageLifecycle,
args=(module, lifecycles)
)
thread.start()
registryname = threadname
registryvalue = thread
lifecycles = registerLifecycle (module, lifecycles, registryname, registryvalue)
threadlist.append(thread)
print(threadlist)
print(lifecycles)
return threadnum, threadname, lifecycles, threadlist
-22
View File
@@ -1,22 +0,0 @@
from createdispatch import buildDispatch
from createthread import buildThread
from dispatchstate import buildState
import threading
def dispatchRequest(request, lifecycles):
module, mode = buildDispatch(request)
mode, lifecycles = buildState(module, mode, lifecycles)
if mode == "stop":
return lifecycles
else:
threadnum, threadname, lifecycles, threadlist = buildThread(module, lifecycles)
#print("Active Threads")
#print(threadlist)
return threadnum, threadname, lifecycles, threadlist
-41
View File
@@ -1,41 +0,0 @@
import requests
from lifecycleregistry import registerLifecycle
def filterMode(mode):
if mode in ("continuous", "oneshot", "stop"):
state = mode
return state
else: raise ValueError(f"Unsupported lifecycle mode: {mode}")
def buildState(module, mode, lifecycles):
state = filterMode(mode)
registryvalue = state
registryname = "mode"
if state != "stop":
lifecycles = registerLifecycle(module, lifecycles, registryname, registryvalue)
elif state == "stop":
if not module.name in lifecycles:
print(f"No registered lifecycle: {module.name}")
return None
elif "mode" in lifecycles[module.name] != "stop":
lifecycles = registerLifecycle(module, lifecycles, registryname, registryvalue)
print(f"Stopping lifecycle: {module.name}")
#elif mode == "oneshot":
#registryvalue = {"mode": "oneshot"}
#lifecycles = registerLifecycle(module, lifecycles, registryname, registryvalue)
# else:
#raise ValueError(f"Unsupported lifecycle mode: {mode}")
#state = registryvalue
return state, lifecycles
-25
View File
@@ -1,25 +0,0 @@
import time
from core.execution.operation import buildOperation
from core.execution.action import buildAction
from core import executeContract
def manageLifecycle(module, lifecycles): #lifecycles, state):
#print(lifecycles[module.name]["mode"])
if lifecycles[module.name]["mode"] == "continuous":
while lifecycles[module.name]["mode"] == "continuous":
print(lifecycles)
outputs = executeContract(module, None, None)
#print(lifecycles)
time.sleep(module.sleeptime)
elif lifecycles[module.name]["mode"] == "oneshot":
print(lifecycles)
outputs = executeContract(module, None, None)
#print(lifecycles)
return
-8
View File
@@ -1,8 +0,0 @@
def registerLifecycle(module, lifecycles, registryname, registryvalue):
if module.name not in lifecycles:
lifecycles[module.name] = {}
lifecycles[module.name][registryname] = registryvalue
return lifecycles
-4
View File
@@ -1,4 +0,0 @@
from plugins import rmass
from operation import buildOperation
outputs = buildOperation(rmass.checkavailableoperation)
-4
View File
@@ -1,4 +0,0 @@
from core.manager.manager import buildManager
from plugins import rmass
outputs = buildManager(rmass.manager)
-29
View File
@@ -1,29 +0,0 @@
import socket
import json
from dispatcher import dispatchRequest
#from createthread import buildThread
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("127.0.0.1", 19002))
lifecycles = {}
server.listen()
while True:
connection, address = server.accept()
while True:
data = connection.recv(4096)
request = json.loads(data)
# if not data:
# break
dispatch = dispatchRequest(request, lifecycles)
connection.close()
# server.close()