gustavosantos/SLACK_COORD_RELATORIO
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(v2): redefine AI actionability semantics + version the classifier (AI_CLASSIFIER_VERSION)

Browse Source 
Production-readiness cycle. Classifier was excessively conservative (real
Run A: ACTIONABLE_NOW/UNASSIGNED_ACTIONABLE both 0) and made concrete
semantic mistakes on real tickets:
- #34419: technician "Rafael Brito" (internal) classified as WAITING_VENDOR
  waiting_for -- an internal technician is not a vendor.
- #37389: ticket had concrete open tasks ("Realizar um inventário das
  máquinas", "Alinhar o dia...") but was classified INSUFFICIENT_CONTEXT.
- Generic next_action ("Revisar chamado") repeated across unrelated
  tickets, proving nothing about real actionability.

Rewrote the system prompt (n8n/nodes/prepare-ai-prompt.node.js) with:
practical actionability definition ("suporte consegue fazer algo util
agora", explicit inclusion list); contatar-vs-aguardar distinction with
examples; WAITING_* requires positive evidence (action already taken AND
external dependency still open); technician-vs-vendor distinction citing
the TECNICO context field; STATUS Planejado alone != blocked/scheduled;
SCHEDULED requires an explicit date/window; mandatory chronological
interpretation (most recent apontamento wins over a superseded old
blocker); next_action must be concrete, with explicit accepted/rejected
examples; INSUFFICIENT_CONTEXT bar raised (ask "is there a diagnostic
step available even without full info" first).
n8n/nodes/prepare-ai-prompt.node.test.js: 10 new tests asserting each
rule's text is still present in the prompt (deterministic proxy --
behavioral proof comes from the real DEV run, inspected manually).

logic/classification.js: added AI_CLASSIFIER_VERSION=3 (stamped on every
valid classification); added a deterministic safety net (not just a
prompt hope) -- isWaitingForInternalTechnician() rejects WAITING_VENDOR
when waiting_for matches the ticket's own TECNICO field, routing to a new
fallback_reason 'ai_semantic_conflict' instead of accepting a known-wrong
semantic classification; added next_action genericness rejection
(isGenericNextAction) for ACTIONABLE_NOW/UNASSIGNED_ACTIONABLE only.
n8n/nodes/validate-ai-response.node.js: now passes the ticket into
validateAiClassification() so the technician guard has the facts it needs.

logic/cache.js: added buildCacheKey(contextHash, classifierVersion) ->
"<hash>:v<version>", stored in the same last_context_hash column (no
schema migration) -- bumping AI_CLASSIFIER_VERSION now forces a cache
miss on every ticket without manually clearing the Data Table.
n8n/nodes/decide-needs-ai.node.js / prepare-memoria-analise-record.node.js
updated to compute/persist the composite key.

logic/config.js: AI_MAX_CANDIDATES default raised 30->60 (now equals
CANDIDATE_POOL_SIZE) -- no candidate should go unanalyzed by an
artificial cap now that Run B already proved the cache makes repeat runs
cheap; AI_FALLBACK stays reserved for real provider failures. Also added
SEND_ENABLED (default false) and ENVIRONMENT config fields for the real
Slack send gate (used by logic/slack.js, next commit).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Gustavo
2026-08-24 20:49:54 -03:00
co-authored by Claude Sonnet 5
parent 4534099b50
commit 482b7ef48e
11 changed files with 300 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
logic/classification.js
+65 -4
View File
@@ -4,6 +4,11 @@ const VALID_CLASSIFICATIONS = [
'PROJECT_LONG_TERM', 'DUPLICATE_OR_OBSOLETE', 'INSUFFICIENT_CONTEXT',
];
// Versao do prompt/classificador. Bump manual sempre que o system prompt ou as regras de
// validacao mudarem de forma que mudaria o julgamento da IA -- forca cache miss em TODO
// chamado (ver logic/cache.js -> buildCacheKey), sem precisar apagar a Data Table na mao.
const AI_CLASSIFIER_VERSION = 3;
// O modelo, apesar do schema permitir null de verdade em blocker_type/waiting_for, as vezes
// devolve a STRING "null" (ou variantes equivalentes) em vez do valor JSON null. Normaliza
// aqui, na camada de validacao -- unico lugar que toca a resposta crua da IA.
@@ -16,7 +21,43 @@ function normalizeNullableString(value) {
return trimmed;
}
function validateAiClassification(raw) {
// next_action generico nao prova acionabilidade nenhuma -- ver missao "finalizacao para
// producao", secao 9. So se aplica a ACTIONABLE_NOW/UNASSIGNED_ACTIONABLE: nessas duas
// classes o next_action E a evidencia de que existe um proximo passo concreto sob controle
// do suporte; nas demais classes o campo descreve o que se espera de terceiros, entao uma
// frase curta ("aguardar resposta") e legitima ali.
const ACTIONABLE_CLASSES_REQUIRING_CONCRETE_NEXT_ACTION = new Set(['ACTIONABLE_NOW', 'UNASSIGNED_ACTIONABLE']);
const GENERIC_NEXT_ACTION_PHRASES = new Set([
'revisar chamado', 'revisar o chamado', 'dar andamento', 'dar andamento no chamado',
'analisar situacao', 'analisar a situacao', 'verificar chamado', 'verificar o chamado',
'acompanhar', 'acompanhar chamado', 'acompanhar o chamado',
]);
function normalizeForComparison(text) {
return String(text || '')
.trim()
.toLowerCase()
.normalize('NFD')
.replace(/[̀-ͯ]/g, '') // remove acentos (combining marks) p/ comparar "situação" == "situacao"
.replace(/[.!]+$/g, '');
}
function isGenericNextAction(nextAction) {
return GENERIC_NEXT_ACTION_PHRASES.has(normalizeForComparison(nextAction));
}
// Guarda deterministica: tecnico interno (aparece no campo TECNICO do proprio chamado) nunca
// pode ser tratado como fornecedor externo (WAITING_VENDOR). Ver missao "finalizacao para
// producao", secao 6. Isso nao depende do prompt se comportar bem -- e verificado aqui.
function isWaitingForInternalTechnician(waitingFor, ticket) {
if (!waitingFor || !ticket || !ticket.tecnico_atribuido) return false;
const tecnico = normalizeForComparison(ticket.tecnico_atribuido);
const alvo = normalizeForComparison(waitingFor);
if (!alvo || tecnico === 'sem tecnico atribuido') return false;
return tecnico.includes(alvo);
}
function validateAiClassification(raw, ticket) {
const errors = [];
if (!raw || typeof raw !== 'object') {
return { valid: false, data: null, errors: ['resposta nao e um objeto JSON'] };
@@ -27,10 +68,22 @@ function validateAiClassification(raw) {
if (typeof raw.actionable_today !== 'boolean') errors.push('actionable_today deve ser boolean');
const confidence = Number(raw.confidence);
if (!Number.isFinite(confidence) || confidence < 0 || confidence > 1) errors.push('confidence deve estar entre 0 e 1');
if (typeof raw.next_action !== 'string' || raw.next_action.trim().length === 0 || raw.next_action.length > 200) errors.push('next_action invalido (vazio ou muito longo)');
if (typeof raw.next_action !== 'string' || raw.next_action.trim().length === 0 || raw.next_action.length > 200) {
errors.push('next_action invalido (vazio ou muito longo)');
} else if (
ACTIONABLE_CLASSES_REQUIRING_CONCRETE_NEXT_ACTION.has(raw.classification)
&& isGenericNextAction(raw.next_action)
) {
errors.push(`next_action generico demais para ${raw.classification}: "${raw.next_action}"`);
}
if (raw.evidence != null && !Array.isArray(raw.evidence)) errors.push('evidence deve ser array');
if (typeof raw.reason !== 'string') errors.push('reason deve ser string');
const waitingFor = normalizeNullableString(raw.waiting_for);
if (raw.classification === 'WAITING_VENDOR' && isWaitingForInternalTechnician(waitingFor, ticket)) {
errors.push(`waiting_for ("${waitingFor}") corresponde a tecnico interno do chamado, nao pode ser WAITING_VENDOR`);
}
if (errors.length) return { valid: false, data: null, errors };
return {
@@ -41,7 +94,7 @@ function validateAiClassification(raw) {
actionable_today: raw.actionable_today,
confidence,
blocker_type: normalizeNullableString(raw.blocker_type),
waiting_for: normalizeNullableString(raw.waiting_for),
waiting_for: waitingFor,
next_action: raw.next_action,
suggested_action_type: raw.suggested_action_type ?? 'technical_action',
business_risk: raw.business_risk ?? 'medium',
@@ -49,8 +102,16 @@ function validateAiClassification(raw) {
evidence: Array.isArray(raw.evidence) ? raw.evidence.slice(0, 5) : [],
is_fallback: false,
fallback_reason: null,
classifier_version: AI_CLASSIFIER_VERSION,
},
};
}
module.exports = { VALID_CLASSIFICATIONS, validateAiClassification, normalizeNullableString };
module.exports = {
VALID_CLASSIFICATIONS,
AI_CLASSIFIER_VERSION,
validateAiClassification,
normalizeNullableString,
isGenericNextAction,
isWaitingForInternalTechnician,
};