Opsgenie Integration

Opsgenie integration enables you to receive 9n9s alerts through Atlassian’s powerful incident management platform, with automated escalation, on-call scheduling, and comprehensive incident response workflows.

Quick Setup

1. Create Opsgenie Integration

   • Log into your Opsgenie account
   • Go to Settings > Integrations
   • Search for “API” and create a new API integration
   • Copy the API key

2. Configure in 9n9s

   • Go to Organization Settings > Notification Channels
   • Click “Add Channel” → “Opsgenie”
   • Enter your API key and team settings
   • Click “Create Channel”

3. Test Integration

   • Click “Send Test Alert”
   • Verify alert appears in Opsgenie dashboard
   • Check escalation and notification settings

Prerequisites

Opsgenie Account Requirements

• Opsgenie Account: Free or paid Opsgenie account
• Team Setup: At least one team configured in Opsgenie
• API Access: Permission to create and manage integrations
• User Configuration: Users and schedules configured for escalation

9n9s Requirements

• Organization Admin: Admin role to configure notification channels
• Active Subscription: Opsgenie integration available on Business and Enterprise plans

Setting Up Opsgenie Integration

Step 1: Configure Opsgenie API Integration

1. Access Opsgenie Settings

   • Log into Opsgenie
   • Navigate to Settings in the left sidebar
   • Click “Integrations”

2. Create API Integration

   • Click “Add Integration”
   • Search for “API” and select it
   • Click “Add” to create the integration

3. Configure Integration Settings

   • Name: “9n9s Monitoring Alerts”
   • Teams: Select the teams that should receive alerts
   • Enabled: Ensure the integration is enabled
   • Alert Actions: Configure default actions for incoming alerts

4. Copy API Key

   • Copy the API Key from the integration details
   • Save this key securely for 9n9s configuration
   • Note the Integration URL if needed

Step 2: Add Opsgenie Channel in 9n9s

1. Navigate to Notification Channels

   • Log into app.9n9s.com
   • Go to Organization Settings
   • Click “Notification Channels”

2. Create Opsgenie Channel

   • Click “Add Channel” button
   • Select “Opsgenie” from integration types
   • Enter configuration details

3. Channel Configuration

   • Channel Name: “Opsgenie Incidents” (internal reference)
   • API Key: Paste your Opsgenie API key
   • Default Team: Primary team for alerts (optional)
   • Priority Mapping: Map 9n9s alert levels to Opsgenie priorities
   • Responders: Default responders for alerts

Step 3: Configure Alert Mapping

1. Priority Mapping

   • Critical (9n9s) → P1 (Opsgenie): Immediate response required
   • High (9n9s) → P2 (Opsgenie): Response within 15 minutes
   • Medium (9n9s) → P3 (Opsgenie): Response within 1 hour
   • Low (9n9s) → P4 (Opsgenie): Response within 4 hours

2. Responder Configuration

   • Default Responders: Teams or users for all alerts
   • Tag-based Routing: Route to specific teams based on monitor tags
   • Time-based Routing: Different responders for business hours vs. after hours
   • Escalation Integration: Use Opsgenie’s escalation policies

Alert Customization

Alert Fields Mapping

9n9s sends the following information to Opsgenie:

Required Fields:

• Message: Alert title (monitor name and status)
• Description: Detailed alert information
• Priority: Mapped from 9n9s alert severity
• Source: Always “9n9s”

Optional Fields:

• Alias: Unique identifier for deduplication
• Tags: Monitor tags from 9n9s
• Details: Additional context and metadata
• Actions: Custom actions for the alert

Message Templates

Critical Alert Example:

{
    "message": "🚨 CRITICAL: API Health Check is DOWN",
    "description": "Monitor 'API Health Check' has been DOWN for 5 minutes.\n\nEnvironment: production\nProject: Core Services\n\nView Monitor: https://app.9n9s.com/monitors/mon_123",
    "priority": "P1",
    "tags": ["9n9s", "api", "production", "critical"],
    "details": {
        "monitor_id": "mon_1234567890",
        "monitor_name": "API Health Check",
        "status": "DOWN",
        "duration": "5 minutes",
        "environment": "production",
        "project": "Core Services"
    },
    "actions": ["View Dashboard", "Check Logs", "Restart Service"]
}

Custom Alert Actions

Configure custom actions that appear in Opsgenie alerts:

{
    "actions": [
        {
            "name": "View 9n9s Dashboard",
            "url": "{{links.monitor}}"
        },
        {
            "name": "Check Application Logs",
            "url": "{{metadata.logs_url}}"
        },
        {
            "name": "View Runbook",
            "url": "{{metadata.runbook_url}}"
        },
        {
            "name": "Escalate to Lead",
            "url": "{{links.escalation}}"
        }
    ]
}

Advanced Configuration

Team-Based Routing

Route alerts to different Opsgenie teams based on monitor characteristics:

routing_rules:
    database_team:
        conditions:
            - tag: "database"
            - environment: "production"
        teams: ["Database Team"]
        priority: "P1"

    frontend_team:
        conditions:
            - tag: "frontend"
            - service: "web"
        teams: ["Frontend Team"]
        priority: "P2"

    default:
        teams: ["Platform Team"]
        priority: "P3"

Escalation Integration

Leverage Opsgenie’s escalation policies:

1. Policy Association

   • Associate specific escalation policies with 9n9s alerts
   • Use monitor tags to determine escalation rules
   • Automatic policy selection based on severity

2. Schedule Integration

   • Route alerts to on-call schedules
   • Time-based team assignments
   • Holiday and maintenance period handling

3. Custom Escalation

   • Create 9n9s-specific escalation policies
   • Multi-level escalation based on response time
   • Cross-team escalation for critical services

Alert Deduplication

Configure intelligent alert deduplication:

{
    "deduplication_config": {
        "alias_template": "9n9s-{{monitor.id}}-{{status}}",
        "auto_close": true,
        "close_after": "300s",
        "update_existing": true
    }
}

Deduplication Strategy:

• Same Monitor, Same Status: Update existing alert
• Status Change: Close previous, create new alert
• Auto-Recovery: Automatically close alerts when monitor recovers
• Time-based Closure: Close alerts after specified duration

Security and Compliance

API Key Security

1. Key Management

   • API keys encrypted at rest in 9n9s
   • Regular key rotation recommended
   • Scope keys to minimum required permissions

2. Access Control

   • Limit API key permissions in Opsgenie
   • Use team-specific keys when possible
   • Monitor API key usage and access patterns

Audit and Compliance

1. Alert Tracking

   • Full audit trail of alerts sent to Opsgenie
   • Integration logs and error tracking
   • Alert delivery confirmation

2. Data Handling

   • Monitor data sent to Opsgenie follows privacy policies
   • No sensitive information in alert messages
   • Compliance with data retention requirements

Troubleshooting

Common Issues

Alerts Not Appearing in Opsgenie:

1. API Key Issues

   • Verify API key is correct and active
   • Check API key permissions in Opsgenie
   • Ensure integration is enabled

2. Team Configuration

   • Verify teams are configured correctly
   • Check team member assignments
   • Ensure team schedules are active

3. Rate Limiting

   • Check Opsgenie API rate limits
   • Monitor alert frequency from 9n9s
   • Verify account subscription limits

Incorrect Alert Priority:

1. Priority Mapping

   • Review priority mapping configuration
   • Check 9n9s alert severity levels
   • Verify Opsgenie priority settings

2. Tag-based Routing

   • Verify monitor tags are correct
   • Check routing rule conditions
   • Test with simplified routing rules

Escalation Not Working:

1. Policy Configuration

   • Verify escalation policies are active
   • Check policy team assignments
   • Ensure schedule coverage

2. Notification Settings

   • Check individual user notification preferences
   • Verify contact methods (email, SMS, push)
   • Test escalation manually in Opsgenie

Error Codes

Common Opsgenie API error responses:

• 401 Unauthorized: Invalid API key
• 403 Forbidden: Insufficient permissions
• 404 Not Found: Team or integration not found
• 422 Unprocessable Entity: Invalid alert data
• 429 Too Many Requests: Rate limit exceeded

Testing and Validation

Test your Opsgenie integration:

# Test Opsgenie API directly
curl -X POST \
  -H "Authorization: GenieKey YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Test alert from 9n9s",
    "description": "This is a test alert",
    "priority": "P3",
    "tags": ["test", "9n9s"]
  }' \
  "https://api.opsgenie.com/v2/alerts"

Best Practices

Alert Design

1. Clear Messaging

   • Use descriptive alert titles
   • Include relevant context in descriptions
   • Provide actionable information

2. Proper Prioritization

   • Reserve P1 for true emergencies
   • Use P2-P4 appropriately for different severities
   • Consider business impact in priority assignment

Team Organization

1. Team Structure

   • Align Opsgenie teams with service ownership
   • Use clear team naming conventions
   • Maintain up-to-date team memberships

2. Schedule Management

   • Ensure adequate on-call coverage
   • Plan for holidays and time off
   • Test schedule rotations regularly

Integration Maintenance

1. Regular Testing

   • Test integration monthly
   • Verify escalation paths
   • Check team assignments and schedules

2. Monitoring Health

   • Monitor alert delivery success rates
   • Track response times and escalations
   • Review and update routing rules

Incident Response

1. Response Procedures

   • Document standard response procedures
   • Train team members on Opsgenie workflows
   • Practice incident response regularly

2. Post-Incident Review

   • Review alert effectiveness
   • Analyze response times and escalations
   • Improve processes based on lessons learned

---

Opsgenie integration provides enterprise-grade incident management capabilities for your 9n9s monitoring. Configure teams, escalation policies, and schedules to ensure critical alerts reach the right people at the right time with appropriate urgency.