Slack Integration

Slack integration allows you to receive 9n9s alerts directly in your team channels with rich formatting, interactive buttons, and thread management. This guide covers both Slack App installation (recommended) and webhook setup methods.

Quick Setup

Method 1: Slack App (Recommended)

1. In 9n9s Dashboard

   • Go to Organization Settings > Notification Channels
   • Click “Add Channel” → “Slack”
   • Click “Install Slack App”

2. Authorize in Slack

   • You’ll be redirected to Slack
   • Select your workspace
   • Choose which channels 9n9s can access
   • Click “Allow”

3. Configure in 9n9s

   • Return to 9n9s (automatic redirect)
   • Select target channel from dropdown
   • Choose message format and options
   • Click “Create Channel”

4. Test Integration

   • Click “Send Test Message”
   • Verify message appears in Slack channel
   • Check formatting and functionality

Prerequisites

Slack Workspace Requirements

• Admin Access: You need admin permissions in your Slack workspace
• Channel Access: Ensure the target channel exists and is accessible
• Workspace Type: Works with free and paid Slack workspaces
• App Permissions: Workspace must allow third-party app installations

9n9s Requirements

• Organization Admin: You need admin or owner role in 9n9s
• Active Subscription: Some advanced features require paid plans
• Project Access: Ensure you have access to projects you want to monitor

Method 1: Slack App Installation (Recommended)

Step 1: Install the 9n9s Slack App

1. Navigate to Notification Channels

   • Log into app.9n9s.com
   • Go to Organization Settings in the sidebar
   • Click “Notification Channels”

2. Start Slack Integration

   • Click “Add Channel” button
   • Select “Slack” from the integration types
   • Click “Install Slack App” button

3. Slack Authorization Process

   • You’ll be redirected to slack.com
   • Sign in to your Slack workspace if not already logged in
   • Review the permissions requested by 9n9s:
     • Send messages to channels
     • Read channel information
     • Access user profile information
   • Click “Allow” to authorize the app

Step 2: Configure Channel Settings

1. Select Target Channel

   • Choose from available channels in your workspace
   • Create a dedicated #alerts or #monitoring channel (recommended)
   • You can select multiple channels for different alert types

2. Message Configuration

   • Message Format: Choose between Compact, Standard, or Detailed
   • Thread Replies: Enable to keep follow-up messages in threads
   • Emoji Reactions: Use status emojis for quick visual status
   • Rich Formatting: Enable Slack blocks and attachments

3. Advanced Settings

   • Mention Settings: Configure when to use @here or @channel
   • Critical Alert Mentions: Automatically mention for critical alerts
   • Quiet Hours: Suppress mentions during specified hours
   • Rate Limiting: Control message frequency (default: 5 per minute)

Step 3: Test and Verify

1. Send Test Message

   • Click “Send Test Message” in the channel configuration
   • Choose test alert severity (Info, Warning, Critical)
   • Verify message appears correctly in Slack

2. Verify Features

   • Check message formatting and colors
   • Test interactive buttons (if enabled)
   • Verify thread functionality
   • Confirm emoji reactions work

Method 2: Webhook Integration

If you prefer not to install the Slack app or need custom webhook handling:

Step 1: Create Slack Webhook

1. Access Slack App Directory

   • Go to your Slack workspace settings
   • Navigate to Apps → Manage
   • Search for “Incoming Webhooks”

2. Configure Incoming Webhook

   • Click “Add to Slack”
   • Select the target channel
   • Customize the webhook name and icon (optional)
   • Click “Add Incoming Webhooks Integration”

3. Copy Webhook URL

   • Copy the provided webhook URL
   • Save it securely (you’ll need it for 9n9s configuration)

Step 2: Add Webhook in 9n9s

1. Create Webhook Channel

   • In 9n9s, go to Organization Settings > Notification Channels
   • Click “Add Channel” → “Slack Webhook”

2. Configure Webhook Settings

   • Channel Name: “Slack Alerts” (or descriptive name)
   • Webhook URL: Paste the URL from Slack
   • Channel: #alerts (for display purposes)
   • Username: “9n9s Monitor” (appears as message sender)
   • Icon: Custom emoji or icon URL (optional)

3. Message Template

   • Choose from predefined templates or create custom
   • Configure color coding for different alert types
   • Set up field mappings for structured data

Message Customization

Message Formats

Compact Format:

🚨 Production API is DOWN (5 minutes)
View Monitor | View Logs

Standard Format:

🚨 ALERT: Production API Health Check
Status: DOWN (was UP)
Duration: 5 minutes
Environment: production
Project: Core Services

View Monitor | View Logs | Acknowledge

Detailed Format:

🚨 CRITICAL ALERT

Service: Production API Health Check
Status: DOWN (was UP)
Project: Core Services
Duration: 5 minutes

Details:
• Environment: production
• Last successful check: 5 minutes ago
• Response code: 500 Internal Server Error
• Response time: 5000ms (exceeded threshold)

Recent Logs:
[14:30:15] HTTP 500 - Database connection timeout
[14:29:45] HTTP 200 - Request successful
[14:29:30] HTTP 200 - Request successful

Actions:
View Monitor | View Logs | Acknowledge | Create Incident

Color Coding

9n9s automatically applies color coding based on alert severity:

• 🟢 Green: UP, Recovered, Resolved
• 🟡 Yellow: DEGRADED, Warning, Late
• 🔴 Red: DOWN, Critical, Failed
• 🔵 Blue: Maintenance, Scheduled
• ⚫ Gray: Paused, Unknown

Interactive Features

When using the Slack App method, you get interactive features:

Action Buttons:

• View Monitor: Direct link to monitor dashboard
• View Logs: Quick access to recent monitor logs
• Acknowledge: Acknowledge the alert (stops escalation)
• Create Incident: Create incident in integrated tools
• Snooze: Temporarily silence alerts for this monitor

Thread Management:

• Follow-up alerts are posted as replies in threads
• Recovery notifications update the original thread
• Status changes are reflected with emoji reactions

Advanced Configuration

Multiple Channels

Set up different channels for different alert types:

1. Critical Alerts Channel

   • Channel: #critical-alerts
   • Conditions: criticality=critical
   • Mentions: @here for immediate attention

2. General Alerts Channel

   • Channel: #alerts
   • Conditions: All other alerts
   • Mentions: None or specific users

3. Team-Specific Channels

   • Channel: #backend-alerts
   • Conditions: team=backend
   • Mentions: Backend team members

Custom Message Templates

Create custom templates for specific use cases:

{
    "blocks": [
        {
            "type": "header",
            "text": {
                "type": "plain_text",
                "text": "🚨 {{alert.severity}} Alert"
            }
        },
        {
            "type": "section",
            "fields": [
                {
                    "type": "mrkdwn",
                    "text": "*Monitor:*\n{{monitor.name}}"
                },
                {
                    "type": "mrkdwn",
                    "text": "*Status:*\n{{monitor.status}}"
                },
                {
                    "type": "mrkdwn",
                    "text": "*Duration:*\n{{incident.duration}}"
                },
                {
                    "type": "mrkdwn",
                    "text": "*Environment:*\n{{monitor.tags.environment}}"
                }
            ]
        },
        {
            "type": "actions",
            "elements": [
                {
                    "type": "button",
                    "text": {
                        "type": "plain_text",
                        "text": "View Monitor"
                    },
                    "url": "{{links.monitor}}"
                },
                {
                    "type": "button",
                    "text": {
                        "type": "plain_text",
                        "text": "View Logs"
                    },
                    "url": "{{links.logs}}"
                }
            ]
        }
    ]
}

Alert Routing Rules

Configure intelligent routing based on monitor attributes:

routing_rules:
    - name: "Critical Production Alerts"
      conditions:
          tags:
              environment: production
              criticality: critical
      slack_settings:
          channel: "#critical-alerts"
          mention: "@here"
          format: "detailed"

    - name: "Development Alerts"
      conditions:
          tags:
              environment: development
      slack_settings:
          channel: "#dev-alerts"
          mention: false
          format: "compact"

    - name: "Database Team Alerts"
      conditions:
          tags:
              service: database
      slack_settings:
          channel: "#database-team"
          mention: "@database-oncall"
          format: "standard"

Security and Permissions

Slack App Permissions

The 9n9s Slack app requests minimal permissions:

Required Permissions:

• chat:write - Send messages to channels
• channels:read - List available channels
• groups:read - List private channels (if granted access)

Optional Permissions:

• users:read - Read user profiles for @mentions
• reactions:write - Add emoji reactions to messages
• im:write - Send direct messages (for personal alerts)

Security Best Practices

1. Channel Access Control

   • Create dedicated alert channels
   • Limit channel membership to relevant team members
   • Use private channels for sensitive alerts

2. Webhook Security

   • Keep webhook URLs confidential
   • Regenerate webhooks if compromised
   • Use HTTPS-only webhooks

3. Message Content

   • Avoid including sensitive data in alert messages
   • Use links to detailed information instead of inline data
   • Configure appropriate message retention in Slack

Audit and Compliance

• Message Logging: All Slack messages are logged in 9n9s audit trail
• Access Tracking: Monitor who has access to notification channels
• Delivery Confirmation: Track message delivery success/failure
• Retention Policies: Configure message retention per compliance requirements

Troubleshooting

Common Issues

Messages Not Appearing in Slack:

1. Check Channel Configuration

   • Verify channel exists and is accessible
   • Confirm 9n9s bot/app has channel access
   • Check if channel is archived or deleted

2. Verify App Permissions

   • Ensure Slack app is properly installed
   • Check if app permissions were revoked
   • Verify workspace admin hasn’t disabled the app

3. Test Connectivity

   • Use “Send Test Message” feature
   • Check 9n9s notification logs
   • Verify webhook URL is correct (for webhook method)

Formatting Issues:

1. Message Appears But Poorly Formatted

   • Check message template configuration
   • Verify Slack blocks/attachments syntax
   • Test with simpler message format first

2. Missing Interactive Features

   • Ensure using Slack App method (not webhook)
   • Check if interactive components are enabled
   • Verify app has necessary permissions

Performance Issues:

1. Delayed Messages

   • Check Slack’s current status
   • Review rate limiting settings
   • Verify webhook endpoint response times

2. Missing Messages During High Volume

   • Increase rate limiting thresholds
   • Enable message queuing
   • Consider using multiple channels for load distribution

Error Codes

Common Slack API Error Responses:

• channel_not_found - Channel doesn’t exist or bot lacks access
• not_in_channel - Bot needs to be invited to the channel
• rate_limited - Too many messages sent too quickly
• invalid_auth - Authentication token is invalid or expired
• account_inactive - Slack workspace is deactivated

Getting Help

Support Resources:

1. 9n9s Support: Use in-app chat or email [email protected]
2. Slack Documentation: Slack API documentation
3. Community: Join our Discord for peer support
4. Status Pages: Check status.9n9s.com for service issues

Best Practices

Channel Organization

1. Dedicated Alert Channels

   • Create #alerts or #monitoring channels
   • Separate critical from non-critical alerts
   • Use descriptive channel names

2. Team-Based Routing

   • Route alerts to appropriate team channels
   • Use @mentions sparingly to avoid alert fatigue
   • Set up escalation paths for unacknowledged alerts

Message Quality

1. Clear and Actionable

   • Include service name and environment in every alert
   • Provide direct links to dashboards and logs
   • Use consistent formatting and terminology

2. Appropriate Detail Level

   • Use compact format for high-frequency, low-importance alerts
   • Use detailed format for critical alerts requiring immediate action
   • Include relevant context without overwhelming recipients

Maintenance

1. Regular Testing

   • Test notification delivery monthly
   • Verify interactive features still work
   • Check that all team members receive alerts appropriately

2. Keep Configuration Updated

   • Update channel membership as team changes
   • Review and update message templates
   • Adjust rate limiting and routing rules as needed

---

Slack integration provides a powerful way to keep your team informed about system health. Start with the Slack App method for the best experience, and customize message formats and routing rules to match your team’s workflow and needs.