Applied AI Engineer

Location: Fully remote; applications are welcomed from across Canada, with priority given to candidates located in Quebec.

Company

hirify.global builds Excel-native reporting, planning, and automation products for finance teams using Acumatica, Sage Intacct, Business Central, and MYOB.

What you will do

• Build and maintain regression suites for GIQL query generation, ERP writeback correctness, and tool selection.
• Define measurable quality standards for each action type and run structured evaluations before model or prompt changes.
• Instrument Gateway actions for token usage, caching, model, latency, and tenant data.
• Establish cost-per-action baselines and identify expensive customers, query patterns, failures, and retry loops.
• Evaluate new models across providers and recommend migration, parallel operation, or deferral based on price and performance.
• Provide unit-cost and margin-impact analysis for credit pricing and product decisions in partnership with Finance.

Requirements

• Substantial production experience building with LLM APIs, beyond demos or notebooks.
• Experience building an evaluation harness and understanding the limitations of naive evaluation approaches.
• Hands-on experience with tracing, observability, and evaluation tools such as Langfuse and promptfoo.
• Strong understanding of token accounting, context-window management, caching strategies, and data analysis.
• Ability to communicate technical trade-offs clearly to non-technical audiences.
• Proficiency in Python or TypeScript.

Nice to have

• Experience with MCP or other tool-calling frameworks.
• Background in ERP, accounting, or financial systems.
• Exposure to usage-based or credit-based pricing models.
• Experience at a small company with independently defined priorities.

Culture & Benefits

• Fully remote work arrangement across Canada.
• Work on governed and auditable AI write access to financial systems of record.
• Direct visibility into product direction, pricing inputs, and model strategy.
• Inclusive workplace committed to diversity and equal opportunity.