- RuleBook Compiler is an AI agent skill that turns a written policy and past decisions into a structured decision table with priorities, outcomes, reasons, and source clauses.
- A bundled local Python script checks the rules for conflicts, dead rules, and gaps, then replays your historical decisions and lists every disagreement.
- It never silently picks a winner between conflicting rules. A conflict, missing input, or invalid input means a person decides.
- It tests consistency, not correctness. It does not show that a policy is right, fair, or legal, or that past decisions were correct.
- You supply your own AI agent, model provider, and Python 3. It is not designed for decisions about people's rights, employment, credit, health, or safety.
RuleBook Compiler: Turn Written Policies Into Tested Decision Rules
Stop re-arguing the same policy every time a case comes in. RuleBook Compiler turns your written policy and past decisions into a decision table you can check, replay, and reuse, so the same facts get the same answer every time.
Written policies often look clear until someone has to apply them to a real case. A customer requests a refund. A support agent checks the policy. A manager reads an exception differently. Another team makes a similar decision using a different rule. The result is inconsistent decisions, repeated questions, and uncertainty about what the policy actually says.
RuleBook Compiler is an AI agent skill built for exactly that gap. Instead of asking an AI agent to interpret the same policy from scratch every time, it creates explicit rules with priorities, outcomes, reasons, and source clauses. A bundled Python script then checks those rules for conflicts, dead rules, gaps, and disagreements with historical decisions. You can open RuleBook Compiler on Capafy at any point.
What Is RuleBook Compiler?
RuleBook Compiler is a downloadable AI agent skill that converts a written policy into structured decision rules. An AI agent reads the policy and writes the rule files, while a local Python script checks the rules, replays past decisions, and evaluates new cases. Its purpose is to make the logic behind repeatable decisions visible, testable, and easier to review.
It is designed for adults who manage repeatable operational decisions, including support and operations leads, shop owners, agencies, and teams working with internal procedures. The workflow combines two components:
- An AI agent: reads the policy, extracts the decision logic, and writes structured rule files.
- A local Python script: checks the rules, replays past decisions, and evaluates new cases using the compiled decision table.
The package does not include an AI model. You provide your own agent and model provider, along with Python 3. RuleBook Compiler is not intended to replace policy owners or professional judgment.
📘 Ready to turn a policy into rules you can test?
Try RuleBook Compiler →Why Compile a Policy Into Decision Rules?
Policies contain conditions, exceptions, thresholds, and several possible outcomes. When these stay in prose, different people interpret them differently. Compiling a policy turns each condition into a structured field and each decision into a rule with a priority, outcome, reason, and source clause, giving you a decision table that can be reviewed before it is used on new cases.
For example, a refund policy might distinguish between:
- A purchase made within a certain number of days
- An item that is unused
- An item damaged on arrival
- A final-sale item
- A case that requires manager review
The idea is not new. Business teams have long used decision tables for exactly this reason, and the Decision Model and Notation (DMN) standard from the Object Management Group formalizes them. RuleBook Compiler brings the same discipline to everyday written policies.
How Does RuleBook Compiler Work?
RuleBook Compiler works in five steps: the agent extracts the policy into a rules file, converts past decisions into test cases, a Python script checks the rules for logical problems, the script replays history against the new rules, and the workflow produces a decision table with a report covering open questions, assumptions, and limitations.
1. Extract the policy into structured rules
The agent reads the policy and creates a rules.json file containing:
- The facts needed to make a decision and the possible outcomes
- The conditions, priority, and reason for each rule
- Source clauses from the original policy
- A default outcome for cases that match no rule
The agent is instructed to use only thresholds and conditions supported by the policy or supplied cases. If the policy is vague, silent, or contradictory, the skill requires the issue to be listed as an open question rather than invented as a rule. If an interpretation is necessary, the rule is marked as an assumption and included in the summary.
2. Convert past decisions into test cases
The agent creates a cases.json file with the inputs and the outcome previously assigned to each case. The package recommends at least 10 past decisions when available. Before using real cases, replace names, emails, order numbers, account numbers, and other personal details with identifiers such as K1, K2, and K3. Historical decisions are copied as supplied, and the skill does not change past outcomes to make the new rules look consistent.
3. Check the rules for logical problems
The bundled script looks for overlapping rules, contradictory conditions, rules that can never apply, rules that are always overridden, and combinations that fall through to the default outcome. It also checks that rules have source clauses and reason text. It tests the declared decision space rather than relying on a few sample cases.
4. Replay historical decisions
The new rules are compared with the outcomes recorded in cases.json. When a rule produces a different outcome from a past decision, the report lists the disagreement. The agent is explicitly told not to weaken a rule or alter a historical outcome merely to make the replay agree. A disagreement is a result that a policy owner needs to review.
5. Produce a decision table and report
The final output is a decision table, a check summary, replay results, open questions, assumptions, and a list of limitations. The agent can also provide a policy skill template so another compatible agent can call the compiled rules later. The rule set becomes a reusable decision resource rather than a one-time conversation.
A Realistic Decision Table Example
A compiled decision table lists rules in priority order, each with its conditions, outcome, and the policy clause it came from. In this illustrative refund example, final-sale items are denied first, damaged items escalate next, and unused items within 30 days are approved. Anything else falls to the default outcome, escalate, so a gap is visible instead of guessed.
The table below is an illustration of the format, based on the refund policy used throughout this article. It is not output copied from the package, and it is not a recommended refund policy.
| Priority | Conditions | Outcome | Source clause |
|---|---|---|---|
| 1 | is_final_sale is true | deny | "Final-sale items are not refundable." |
| 2 | item_condition is damaged | escalate | "Damaged items require review." |
| 3 | days_since_purchase ≤ 30 and item_condition is unused | approve | "Purchases within 30 days may be refundable." |
| Default | No rule matches | escalate | "Cases that do not match a rule must be escalated." |
Replaying past decisions against the table
Now imagine four anonymized past cases. The replay compares what the compiled rules would decide with what was actually decided.
| Case | Facts | Recorded decision | Compiled outcome | Result |
|---|---|---|---|---|
| K1 | Final sale, 10 days | deny | deny | Match |
| K2 | Damaged, 5 days | escalate | escalate | Match |
| K3 | Unused, 12 days | approve | approve | Match |
| K4 | Unused, 40 days | approve | escalate (default) | Disagreement for review |
K4 is the useful result. A past decision approved a refund outside the 30-day window, which the written policy does not cover. RuleBook Compiler does not "fix" this by editing the rule or the history. It surfaces the disagreement so the policy owner can decide whether the policy, the rule, or the past decision needs attention.
Illustration only. This example shows how a decision table and replay look. It does not establish that any particular refund policy is legally valid or that a particular customer should receive a refund.
What Problems Can RuleBook Compiler Detect?
The bundled checker reports numbered findings, RB-01 to RB-14 (there is no RB-10). They cover same-priority conflicts, rules that can never match, always-overridden rules, default-outcome gaps, missing source clauses or reasons, duplicates, unused fields and outcomes, replay disagreements, unexercised rules, undecidable cases, and too few historical cases. These findings expose structural problems before the table decides new cases.
| Check | What it identifies |
|---|---|
| RB-01 | Two rules can match the same case with the same priority but different outcomes |
| RB-02 | A rule can never match because its conditions contradict each other |
| RB-03 | A rule is always overridden by higher-priority rules |
| RB-04 | Input combinations that match no rule and fall to the default |
| RB-05 | A rule has no source clause |
| RB-06 | Rules with identical conditions and outcomes |
| RB-07 | Declared fields that no rule uses |
| RB-08 | Declared outcomes that no rule produces |
| RB-09 | Rules without reason text |
| RB-11 | Historical cases where the compiled outcome differs from the recorded decision |
| RB-12 | Rules that were not exercised by supplied cases |
| RB-13 | Cases that cannot be decided because of conflicts, missing inputs, or invalid inputs |
| RB-14 | Fewer than 10 historical cases supplied |
The checker also includes a size-related finding for rule sets that are too large for its combination analysis.
How Does RuleBook Compiler Resolve Overlapping Rules?
RuleBook Compiler uses explicit priorities: the highest-priority matching rule produces the outcome. A specific exception, such as denying final-sale items, can outrank a general rule that allows refunds within 30 days. If two matching rules share a priority but disagree on the outcome, it reports a conflict instead of silently choosing one, and a person must review it.
- Different priorities: the higher-priority rule wins, so specific exceptions can override general rules.
- Same priority, same outcome: both rules agree, so this is not a conflict (identical duplicate rules are flagged separately as RB-06).
- Same priority, different outcomes: a conflict that requires human review before the result is relied on.
This distinction matters because a decision system should not hide ambiguous logic behind a confident answer.
Can RuleBook Compiler Find Gaps in a Policy?
Yes. The checker evaluates the input combinations implied by your declared fields and rules. For numeric fields it tests values around each threshold, and for fields with a fixed list of values it tests every declared value plus one outside the list. Combinations that match no rule are reported as gaps that fall to the default outcome, such as escalate.
For example, a refund policy might cover unused items and final-sale items but leave damaged items unaddressed. The checker can identify that the decision table has a gap, but it cannot decide what the policy owner should do about it.
Setting a default outcome such as escalate makes the gap explicit instead of forcing the agent to invent an answer. Note that the share of tested combinations that fall to the default is not a real-world frequency rate.
Find the Gaps Before Your Customers Do
Compile a written policy, check it for conflicts and gaps, and replay your past decisions in one workflow.
Try RuleBook Compiler on Capafy → More AI Agent SkillsChecks consistency · Does not judge fairness or legality
How Can You Use the Compiled Rules for New Decisions?
After the rules have been checked and reviewed, the package's decide command evaluates a single case supplied as a JSON file of facts. It returns the outcome with the applicable rule, reason, and source clause. If inputs are missing or invalid, or the rules conflict, the result requires a person to decide, and the agent reports the script's result rather than overriding it.
- Review the checked rule set and resolve open questions.
- Describe the new case as a JSON file containing the facts the rules require.
- Run the
decidecommand and read the outcome, rule, reason, and source clause. - If the result says a person must decide, escalate it instead of asking the agent to guess.
This is the central difference between compiling a decision process and repeatedly asking an AI agent to interpret a policy from scratch. The same facts produce the same traceable answer, in the same format.
Can RuleBook Compiler Track Changes to Policies?
Yes. The diff command compares an old rule file with a new one. When a policy changes, you compile the revised policy, compare the two decision tables, and see which supplied historical cases would receive different outcomes. It makes the effect of a change visible, but it does not judge whether the change is fair, appropriate, or legally required.
For example, a policy owner could change a refund threshold from 30 days to 45 days and use the diff report to identify the past cases whose outcomes would change under the revised rules. In the example above, K4 (unused, 40 days) is exactly the kind of case that would move.
Policies that touch customer terms may also benefit from a legal read. See how T&C Liability Shield reviews terms before you agree.
What Does the Package Include, and Do You Need API Keys?
The package includes the agent instructions, a Python checker and executor, templates, reference documents, sample policies and cases, expected reports, and a test suite. The script uses only Python's standard library and makes no network calls. However, the package contains no AI model, so you supply your own agent and model provider, and any usage fees come from that provider.
The downloadable package contains:
SKILL.mdwith the agent instructions- A Python decision-rule checker and executor, plus a test suite
- A policy skill template and a decision-log template
- A rule-format reference, a check catalog, and data-handling documentation
- Sample policy files, sample historical cases, and expected reports for a sample policy and a deliberately flawed policy
The documented environment is Python 3, developed and tested on Python 3.13. Other Python 3 versions are not tested in the package documentation. See the Python json module documentation for the format used by the rule and case files.
The policy text and historical cases are processed by whichever AI agent and model provider you use, under that provider's terms. Remove personal details and confidential information first, and do not include passwords, keys, ID numbers, or customer account details.
What RuleBook Compiler Does Not Measure
RuleBook Compiler checks that your rules are consistent and compare them with the cases you supplied. It does not measure whether the policy wording is correct, whether the policy is fair or legal, whether past decisions were right, how often each input combination occurs in real life, or whether the rules match the policy owner's intended interpretation.
- Whether the policy wording is correct
- Whether a policy is fair or legal
- Whether past decisions were correct
- How frequently each input combination occurs in real life
- Cases and input values that were not supplied
- Whether the compiled rules reflect the owner's intended interpretation
A rule set can pass every consistency check and still encode a mistaken interpretation of the policy. That is why the skill requires source clauses, open questions, assumptions, and human review. If you also need to verify claims made in policy or marketing text, see how ClaimLock audits claims against sources.
Important Safety and Use Limitations
RuleBook Compiler is intended for adults aged 18 and older and for operational policies such as refund handling or internal procedures. It is not designed for decisions about people's rights, employment, credit, health, or safety, and it is not legal, financial, or HR advice. The policy owner remains responsible for reviewing rules and resolving ambiguity.
Testable logic, not transferred responsibility. The tool's purpose is to make decision logic testable, not to hand responsibility for consequential decisions to an AI agent. For broader guidance on managing AI risk, see the NIST AI Risk Management Framework.
Who Should Use RuleBook Compiler?
RuleBook Compiler suits customer support and operations teams, shop owners, agencies, and compliance staff who apply a written policy repeatedly and want consistent, explainable outcomes. It works best when you have a written policy, a defined set of outcomes, and enough historical decisions, ideally at least 10, to test whether the compiled rules match existing practice.
- Customer support and operations teams applying refund, escalation, or eligibility policies.
- Shop owners managing repeatable refund policies. Pair it with our ecommerce product launch plan guide.
- Agencies applying internal procedures across clients.
- Compliance staff reviewing operational decision logic. See also ScoreTrace AI for auditing track records.
- AI agent users who want reusable rules instead of repeated policy interpretation.
RuleBook Compiler — Frequently Asked Questions
These answers cover what RuleBook Compiler is, whether it includes an AI model, what it checks, how it handles conflicts, and where its limits are. In short, it turns written policies into checked, replayable decision tables, but it tests consistency only and never judges whether a policy is right, fair, or legal.
Final Thoughts
RuleBook Compiler turns policy interpretation into an explicit decision system. It extracts rules from written policies, checks them for conflicts and gaps, replays past decisions, and produces a reusable decision table. Its most valuable feature is making disagreements, missing conditions, and assumptions visible before the rules are relied upon. It tests consistency, not correctness.
The closing principle is worth remembering: RuleBook Compiler tests that your rules are consistent and match the cases you gave it. It does not show that the policy is right or legal, or that past decisions were correct. Explore more in our AI Agent Skills category and the Free AI Online Tools collection.
Next step: Try RuleBook Compiler on Capafy and see where your written policy and your past decisions disagree.
