Analyze training metrics and detect reward hacking

Use this endpoint to analyze the full training metric history of a job. The analysis detects reward spikes that may indicate reward hacking and computes summary statistics over the entire run. This is useful for auditing RL jobs before accepting a model into production.

Method + Path

GET /v1/jobs/:id/analysis

Authentication

Include your PIN API token or JWT in the Authorization header as a Bearer token.

Authorization: Bearer <token>

Path parameters

string

required

The job identifier (e.g., jb_abc123def456).

Request example

curl -s "https://api.ariacompute.com/v1/jobs/jb_abc123def456/analysis" \
  -H "Authorization: Bearer $PIN_API_TOKEN"

Response

Returns a standard PIN success envelope with the analysis result.

integer

required

0 on success, nonzero on error.

object

required

Show analysis object

object

Reward statistics.

Show properties

number

Mean reward across all steps.

number

Standard deviation of reward.

array

Step numbers where reward spiked (>= 2x the preceding 10-step baseline).

string

Risk assessment: ok or suspected_reward_hacking.

array

List of human-readable evidence strings.

string

Human-readable message (empty on success).

Example response

{
  "code": 0,
  "data": {
    "reward": {
      "mean": 0.7421,
      "std": 0.3124,
      "spike_steps": [1240, 1890]
    },
    "risk": "suspected_reward_hacking",
    "evidence": [
      "reward spiked multiple times (>=2x baseline)",
      "second-half reward mean is >50% above first-half"
    ]
  },
  "message": ""
}

Detection logic

The analyzer applies the following checks over the full step history:

Reward spike detection: For each step after the first 10, if the reward is at least 2x the mean of the preceding 10 steps, that step is flagged as a spike.
Second-half comparison: If the mean reward in the second half of training is more than 50% higher than the first half, evidence is added.
Risk flag: Two or more spike steps trigger suspected_reward_hacking. Otherwise the risk is ok.
High variance warning: If no spikes are found but standard deviation exceeds 0.3, a manual review recommendation is appended.

Non-RL jobs (SFT, OPD) that do not report reward are skipped for reward-specific checks.

Errors

Code	HTTP	Meaning
401	401	Missing or invalid Bearer token.
404	404	Job not found or does not belong to you.
422	422	Job has no metrics yet (training not started or not reported).
500	500	Internal server error.

PIN API

ROUTER API

AGENT API

Analyze training metrics and detect reward hacking

Method + Path

Authentication

Path parameters

Request example

Response

Example response

Detection logic

Errors

​Method + Path

​Authentication

​Path parameters

​Request example

​Response

​Example response

​Detection logic

​Errors

Method + Path

Authentication

Path parameters

Request example

Response

Example response

Detection logic

Errors