Mastering Facebook APIs: A Guide to Media Handling, Encryption, and Messaging

Mastering Facebook APIs: A Guide to Media Handling, Encryption, and Messaging

Facebook's documentation is all over the place. It's quite difficult to specifically find what you're looking for, considering they have multiple pages that seem to leapfrog one another in delightful disarray. Whether it's sending messages, assembling message components, or mastering webhook APIs, the rabbit holes are plentiful. Feel like you’re running in circles? You're not alone.

This guide aims to present a clear path through this jungle of information, focusing specifically on handling media, encryption, and integrating WhatsApp flows. The documentation you need is within one place here, so go on—breathe easy, savor those extra years you’ve gotten back. You’re welcome!

Downloading and Handling Media

There's two ways to handle media when navigating the labyrinth of Facebook APIs: under WhatsApp flows and using the webhook API. We'll start off by easing into the mystical world of the webhook API, and then gradually move towards the more complicated WhatsApp flows.

1. Downloading and Handling Media (webhook API)

This is as simple as the code snippet below. If you'd like to explore further, here's the documentation.

protected async downloadMedia(mediaId: string) {
  const env = getEnvs();
  const headers = { Authorization: `Bearer ${this.accessToken}` };
  const mediaUrl = await lastValueFrom(this.httpService.get(`https://graph.facebook.com/${env.env.whatsappConfig.version}/${mediaId}`, { headers }));
  
  if (mediaUrl?.data?.['url']) {
    const response = await lastValueFrom(this.httpService.get(mediaUrl?.data?.['url'], {
      headers,
      responseType: 'arraybuffer',
    }));
    const file: Buffer = response.data;
    return Buffer.from(file).toString('base64');
  }
  return undefined;
}

This block of code directly accesses and downloads media files, transforming them into base64 format for easy handling.

2. Downloading and Handling Media (WhatsApp Flows)

Now, let's talk about handling media in WhatsApp flows. This method involves making sure the media is securely encrypted and checked for integrity. Here’s a quick overview of the process:

  1. Download the media from the CDN using the URL provided.
  2. Verify the media hash to confirm it hasn't been tampered with.
  3. Validate the HMAC to further ensure the integrity of the media content.
  4. Decrypt the media file using AES-256-CBC, and verify its hash matches the expected hash.

You first need to generate a private-public key pair as explained here. You also need to set up the necessary callbacks as described in this documentation.

For set up, here's an example snippet that should provide you with a blueprint/framework of the process.

@Injectable()
export class WhatsappBotUtilsService extends BaseService {
  protected readonly whatsappApi: string;
  protected readonly accessToken: string;
  private readonly verificationToken: string;
  private readonly whatsappMessagingApi: string;

  constructor(
    protected readonly httpService: HttpService,
    ) {
    super();
    const { env } = getEnvs();
    this.verificationToken = env.whatsappConfig.verificationToken;
    this.accessToken = env.whatsappConfig.accessToken;
    this.whatsappApi = `https://graph.facebook.com/${env.whatsappConfig.version}/${env.whatsappConfig.phoneNumberId}`;
    this.whatsappMessagingApi = `${this.whatsappApi}/messages`;
  }

  async verify(token: string, challenge: string): Promise<string> {
    if (token === this.verificationToken) {
      return challenge;
    }
    return 'error';
  }

  public async sendMessage(message: WhatsAppOutgoingMessage, apiOpts?: {
    phoneNumberId?: string,
    accessToken?: string
  }): Promise<void> {
    if (message.to.includes('+')) {
      message.to = message.to.replace('+', '');
    }
    const { env } = getEnvs();

    const api = apiOpts?.phoneNumberId ? `https://graph.facebook.com/${env.whatsappConfig.version}/${apiOpts.phoneNumberId}/messages` : this.whatsappMessagingApi;
    console.info('api: ', api);
    console.info('API Opts: ', apiOpts);
    console.info('outgoing message: ', message);

    await firstValueFrom(
      this.httpService.post(
        api,
        message,
        {
          headers: {
            Authorization: `Bearer ${apiOpts?.accessToken || this.accessToken}`,
            'Content-Type': 'application/json',
          },
        },
      ).pipe(
        catchError((error: AxiosError) => {
          throw error;
        }),
      ),
    ).catch((error: AxiosError) => {
      console.log('error: ', error.message, error.response?.data);
    });
  }

  protected async downloadMedia(mediaId: string) {
    const env = getEnvs();
    const headers = {
      Authorization: `Bearer ${this.accessToken}`,
    };
    const mediaUrl = await lastValueFrom(this.httpService.get(`https://graph.facebook.com/${env.env.whatsappConfig.version}/${mediaId}`, { headers }));
    if (mediaUrl?.data?.['url']) {
      const response = await lastValueFrom(this.httpService.get(mediaUrl?.data?.['url'], {
        headers,
        responseType: 'arraybuffer',
      }));
      const file: Buffer = response.data;
      return Buffer.from(file).toString('base64');
    }
    return undefined;
  }

}

Lastly, for actual media encryption and decription, here's a code snippet that does all of this for you (powered by nestjs), but again, feel free to read more here.

@Injectable()
export class WhatsappFlowEncryptionService {
  private readonly appSecret;
  private readonly privateKey;
  private readonly passphrase;

  constructor(protected readonly httpService: HttpService) {
    const { env } = getEnvs();
    this.appSecret = env.whatsappConfig.secret;
    this.privateKey = env.whatsappConfig.privateKey;
    this.passphrase = env.whatsappConfig.passphrase;
  }


  public decryptRequest = (req: RawBodyRequest<Request>): WhatsAppFlowDecryptedRequest => {
    if (!this.isRequestSignatureValid(req)) {
      throw new WhatsAppFlowException('The request signature is invalid', 432);
    }
    if (!this.privateKey) {
      throw new WhatsAppFlowException('No private key defined.', 421);
    }

    const { encrypted_aes_key, encrypted_flow_data, initial_vector } = req.body;

    let decryptedAesKey = null;
    try {
      const privateKey = crypto.createPrivateKey({
        key: this.privateKey || '',
        passphrase: this.passphrase,
      });

      // decrypt AES key created by client
      decryptedAesKey = crypto.privateDecrypt(
        {
          key: privateKey,
          padding: crypto.constants.RSA_PKCS1_OAEP_PADDING,
          oaepHash: 'sha256',
        },
        Buffer.from(encrypted_aes_key, 'base64'),
      );
    } catch (error) {
      console.error('DECRYPT AES KEY ERROR:', error, '\n', '\n');
      /*
      Failed to decrypt. Please verify your private key.
      If you change your public key. You need to return HTTP status code 421 to refresh the public key on the client
      */
      throw new WhatsAppFlowException(
        'Failed to decrypt the request. Please verify your private key.',
        421,
      );
    }

    // decrypt flow data
    const flowDataBuffer = Buffer.from(encrypted_flow_data, 'base64');
    const initialVectorBuffer = Buffer.from(initial_vector, 'base64');

    const TAG_LENGTH = 16;
    const encrypted_flow_data_body = flowDataBuffer.subarray(0, -TAG_LENGTH);
    const encrypted_flow_data_tag = flowDataBuffer.subarray(-TAG_LENGTH);


    let decryptedJSONString = '{}';
    try {
      const decipher = crypto.createDecipheriv(
        'aes-128-gcm',
        decryptedAesKey,
        initialVectorBuffer,
      );
      decipher.setAuthTag(encrypted_flow_data_tag);

      decryptedJSONString = Buffer.concat([
        decipher.update(encrypted_flow_data_body),
        decipher.final(),
      ]).toString('utf-8');
    } catch (error) {
      console.error('DECRYPT AES KEY ERROR:', error, '\n');
      /*
      Failed to decrypt. Please verify your private key.
      If you change your public key. You need to return HTTP status code 421 to refresh the public key on the client
      */
      throw new WhatsAppFlowException(
        'Failed to decrypt the request. Please verify your private key.',
        421,
      );
    }

    return {
      decryptedBody: JSON.parse(decryptedJSONString),
      aesKeyBase64: decryptedAesKey.toString('base64'),
      initialVectorBase64: initialVectorBuffer.toString('base64'),
    };
  };

  public encryptResponse = (
    response: WhatsAppFlowDecryptedResponse,
  ) => {
    const { aesKeyBase64, initialVectorBase64, decryptedBody } = response;

    // flip initial vector
    const flipped_iv = [];
    for (const pair of Buffer.from(initialVectorBase64, 'base64').entries()) {
      flipped_iv.push(~pair[1]);
    }

    // encrypt response data
    const cipher = crypto.createCipheriv(
      'aes-128-gcm',
      Buffer.from(aesKeyBase64, 'base64'),
      Buffer.from(flipped_iv),
    );
    return Buffer.concat([
      cipher.update(JSON.stringify(decryptedBody), 'utf-8'),
      cipher.final(),
      cipher.getAuthTag(),
    ]).toString('base64');
  };

  public async decryptMedia(jsonPayload: WhatsAppResponseMedia) {
    const {
      cdn_url,
      encryption_metadata: {
        encrypted_hash,
        iv,
        encryption_key,
        hmac_key,
        plaintext_hash,
      },
    } = jsonPayload;

    // Step 1: Download cdn_file from cdn_url
    const response = await lastValueFrom(this.httpService.get(cdn_url, { responseType: 'arraybuffer' }));
    const cdn_file: Buffer = response.data;

    // Step 2: Make sure SHA256(cdn_file) == enc_hash
    const cdn_file_hash = crypto.createHash('sha256').update(cdn_file).digest('base64');
    if (cdn_file_hash !== encrypted_hash) {
      console.error('CDN file hash mismatch', '\n');
      throw new WhatsAppFlowException('CDN file hash mismatch', 500);
    }

    // Step 3: Validate HMAC-SHA256
    const threshold = cdn_file.length - 10;

    // since file structure = cipherText + hmac10
    const _hmac10 = Buffer.from(cdn_file);
    const hmac10 = _hmac10.subarray(threshold);

    const _cipherText = Buffer.from(cdn_file);
    const cipherText = _cipherText.subarray(0, threshold);

    // Calculate HMAC with hmac_key, initialization vector (encryption_metadata.iv) and cipherText
    const hmac = crypto.createHmac('sha256', Buffer.from(hmac_key, 'base64'));
    hmac.update(Buffer.from(iv, 'base64'));
    hmac.update(cipherText);

    const calculated_hmac = hmac.digest();

    // Make sure first 10 bytes == hmac10
    const first10Bytes = Buffer.from(calculated_hmac.subarray(0, 10)); // From the calculated hmac.

    if (Buffer.compare(hmac10, first10Bytes) !== 0) {
      console.error('HMAC validation failed', '\n');
      throw new WhatsAppFlowException('HMAC validation failed', 500);
    }

    // Step 4: Decrypt media content:

    // Run AES with CBC mode and initialization vector (encryption_metadata.iv) on cipherText
    const decipher = crypto.createDecipheriv(
      'aes-256-cbc',
      Buffer.from(encryption_key, 'base64'),
      Buffer.from(iv, 'base64'),
    );

    // Remove padding (AES256 uses blocks of 16 bytes, padding algorithm is pkcs7) - ode.js does this automatically
    const decrypted_media = Buffer.concat([decipher.update(cipherText), decipher.final()]);

    // Step 5: Validate the decrypted media

    // Make sure SHA256(decrypted_media) = plaintext_hash
    const decrypted_media_hash = crypto.createHash('sha256').update(decrypted_media).digest('base64');
    if (decrypted_media_hash !== plaintext_hash) {
      console.error('Decrypted media hash mismatch');
      throw new Error('Decrypted media hash mismatch');
    }

    return decrypted_media.toString('base64');
  }

  private isRequestSignatureValid(req: RawBodyRequest<Request>) {
    if (!this.appSecret) {
      console.warn('App Secret is not set up. Please Add your app secret in /.env file to check for request validation', '\n');
      return true;
    }

    const signatureHeader = req.get('x-hub-signature-256');
    if (!signatureHeader) {
      console.warn('No x-hub-signature-256 header', '\n');
      return false;
    }
    const signatureBuffer = Buffer.from(signatureHeader.replace('sha256=', ''), 'utf-8');

    const hmac = crypto.createHmac('sha256', this.appSecret);
    const digestString = hmac.update(req.rawBody?.toString() || '').digest('hex');
    const digestBuffer = Buffer.from(digestString, 'utf-8');

    if (!crypto.timingSafeEqual(digestBuffer, signatureBuffer)) {
      console.error('Error: Request Signature did not match', '\n');
      return false;
    }
    return true;
  }
}

By employing both of these methods, you'll be well-equipped to manage media efficiently across different platforms.

Messaging and Error Handling

Managing messages and handling errors effectively is crucial in working with Facebook APIs. You've already seen some of this in the previous code snippets, so I'll break this down here to it's constituent methods for a better look at how to handle incoming messages and address errors.

Handling Incoming Messages

When a new WhatsApp message comes in, it’s important to process it efficiently. Below is a simple process to manage this:

async handleIncomingWhatsAppMessage(message: WhatsAppIncomingMessage) {
  console.log('Handling incoming WhatsApp message', JSON.stringify(message), '\n\n');
  
  const entry = message.entry[0];
  const messages = entry.changes.find(entry => entry.field === 'messages')?.value?.messages || [];

  if (await this.farmerOnboardingStoryService.canHandleMessage({ message, actualMessage: messages })) {
    return this.farmerOnboardingStoryService.handleMessage({ message, actualMessage: messages });
  }
  // Further checks and error handling...
}

async verify(token: string, challenge: string): Promise<string> {
    if (token === this.verificationToken) {
      return challenge;
    }
    return 'error';
  }

  public async sendMessage(message: WhatsAppOutgoingMessage, apiOpts?: {
    phoneNumberId?: string,
    accessToken?: string
  }): Promise<void> {
    if (message.to.includes('+')) {
      message.to = message.to.replace('+', '');
    }
    const { env } = getEnvs();

    const api = apiOpts?.phoneNumberId ? `https://graph.facebook.com/${env.whatsappConfig.version}/${apiOpts.phoneNumberId}/messages` : this.whatsappMessagingApi;
    console.info('api: ', api);
    console.info('API Opts: ', apiOpts);
    console.info('outgoing message: ', message);

    await firstValueFrom(
      this.httpService.post(
        api,
        message,
        {
          headers: {
            Authorization: `Bearer ${apiOpts?.accessToken || this.accessToken}`,
            'Content-Type': 'application/json',
          },
        },
      ).pipe(
        catchError((error: AxiosError) => {
          throw error;
        }),
      ),
    ).catch((error: AxiosError) => {
      console.log('error: ', error.message, error.response?.data);
    });
  }
  1. The verify method
  • This method ensures that the incoming requests are valid and originate from a trusted source, protecting your application from potential spoofing or unauthorized access.
  • token: This is the token that you've configured with WhatsApp to ensure that your app is what it claims to be. It acts as a shared secret between WhatsApp and your application.
  • challenge: A unique string sent by WhatsApp, which you must return to validate the request. This is unique for each request sent.
  1. The handleIncomingWhatsAppMessage method
  • This method acts as a traffic controller for incoming messages, deciding which service should handle each message based on predefined conditions.
  • Logging: It starts by logging the entire message for debugging and auditing purposes.
  • Extracting Entries: The method retrieves the entry array from the message, isolating the changes related to incoming messages.
  • Message Filtering: It looks for changes that are specifically concerned with messages using the entry field messages.
  • Routing Logic: It checks each message against various conditions using different services designed to handle specific types of messages. For instance, if a message can be processed by the farmerOnboardingStoryService, the method routes the message to this service. Add more conditions as required by your application's logic to handle other story services.

Handling Errors

Handling errors is essential to ensure robust and resilient application behavior.

NB: Always make sure to catch all errors, and respond with a 200 OK response to WhatsApp, otherwise the messages will be considered failed, and will continously retry until WA servers get a 200 OK response. This is a bad experience for your users if a lot of messages have been queued since they will all be sent once the error is resolved, and will lower the quality rating of your account. Yes, I had to learn this the hard way unfortunately.

Here's how you can catch and manage errors:

@Catch(HttpException)
export class RESTHttpExceptionFilter implements ExceptionFilter {
  catch(exception: HttpException, host: ArgumentsHost) {
    const ctx = host.switchToHttp();
    const response = ctx.getResponse<Response>();
    const request = ctx.getRequest<Request>();
    const res = new GqStandardEmptyResponse();
    res.status = exception.getStatus();
    console.error('HTTPExceptionFilter exception: ', exception);

    if (exception instanceof WhatsAppHandleMessageException) {

      // Here, we always return a 200 response to prevent WA servers from queing these messages to improve user experience.
      return response.status(200);
    }

    switch (res.status) {
      case 401:
        res.message = getTranslations(request.body.preferredLanguage)['not_authorized'];
        break;

      default:
        res.message = getTranslations(request.body.preferredLanguage)['server_error'];
        break;
    }

    return response.status(res.status).json(res);
  }
}

Use Cases: Where Our Integration Shines

With your newfound knowledge of Facebook API integration, let's explore some practical scenarios where these techniques can truly make an impact:

  1. Enhanced Customer Support: By efficiently handling incoming WhatsApp messages, businesses can provide timely and personalized support to their customers. This ensures happier customers and improved service ratings.

  2. Automated Notifications: Automate sending out notifications for order confirmations, shipping updates, or appointment reminders—saving time and ensuring your customers are always informed.

  3. Secure Media Handling: Utilize secure methods for downloading and encrypting media through WhatsApp flows to protect sensitive information and maintain customer trust.

  4. Dynamic Content Distribution: Distribute personalized offers or targeted messages at scale using the flexibility provided by WhatsApp and Facebook integrations.

Conclusion

Congratulations, you've turned a potentially overwhelming task into a suite of manageable, actionable components. By effectively handling media, ensuring secure message exchanges through encryption, and competently managing incoming messages, you’ve enhanced your app's interactions and data security.

Whether you're building sophisticated chatbots or simply improving customer communication, these tools and techniques empower you to leverage Facebook's rich API offerings fully. Now, it's time to put this knowledge into practice. How will you implement these methods in your project? Share your thoughts and questions in the comments below—we’re all here to learn and grow together!

Similar Posts