Laravel and NodeJS messaging using Redis Pub/Sub

I was recently working on this project that was composed of two different parts: a web application built in PHP with Laravel, and an AWS Lambda function written in NodeJS. In the past, both applications exchanged data using a common MySQL database. With time, this setup showed up very inefficient. As the number of “messages” sent and received increased, the database started to not handling well the volume of reads and writes required to support both “applications” — the Lambda function is not an application per se but you know what I mean, right?

The first thing we tried was changing the database schema to focus on performance, rather than on data integrity. We dropped some constraints and changed how the data was stored to achieve that. The updates soon showed themselves not enough.

In a second iteration, we started playing around with Redis. Due to its nature, a key/value store and not a relational database, it’s a lot faster than MySQL. The first attempt using Redis involved simply moving the data we’re storing into the database to a set. It seemed to work well but just after a few tests on a staging server we realized that approach wouldn’t work for the system needs. When retrieving the data using the SCAN command, the order of returned elements is not guaranteed. And that was an important downside for us, the business logic required us to read the data in the same order it was written.

Finally, we got to the setup we have now: both sides — the web app and the Lambda function — were updated to use Redis Pub/Sub implementation. Laravel supports Redis out of the box, which was a nice thing to have. For the NodeJS part, we used NodeRedis.

Subscribing to a channel

As I mentioned, Laravel already has an interface to deal with Redis. It still needs an underlying client, but most of the operations are pretty straightforward. You may refer to the Laravel docs for more info. Subscribing to a channel requires a single method call:

Redis::subscribe([ 'channel_name' ], function ($message) {
    /* Do whatever you need with the message */
}

I’m using an Artisan command to start this listener, this way:

class Subscriber extends Command
{
    protected $signature = 'redis:subscriber';

    protected $description = '...';

    public function handle()
    {
        Redis::subscribe([ 'channel_name' ], function ($message) {
            $this->processMessage($message);
        });
    }

    public function processMessage(string $message)
    {
        /* Handles the received message */
        $this->info(sprintf('Message received: %s', $message));
    }
}

Now we simply have to trigger the command to start listening to the channel.

You’ll notice that after a minute without receiving any data, the next time the subscriber gets a message an error will be thrown. That’s because the connection timed out. To fix that, we added the following settings to the config/database.php file, inside the "redis" block:

'read_write_timeout' => 0,
'persistent' => 1,

Publishing to the channel

On the NodeJS side, we need the aforementioned library. To install it:

$ npm install redis

After that, we’ll need to write our Lambda function that publishes to the channel. Since the focus is the Pub/Sub flow, I’m not using any particular logic to create the message here, just returning the attribute received with the event.

const redis = require('redis');
const client = redis.createClient();

const handler = (event, context) => {
    const message = processEvent(event);
    client.publish('channel_name', message);
    return context.done(null, {
        message,
    });
};

const processEvent = (event) => {
    /* Handles the event and return the message to publish */
    return event.message;
};

exports.handler = handler;

Notice I’m not passing any properties to the createClient function. You’ll probably want to set the host or any other custom configuration you have to properly connect to the Redis instance. Check the NodeRedis docs for more info about the available properties.

Testing all together

First, start the Artisan command. If you used the same name from my example above, you should be able to run the following:

$ php artisan redis:subscriber

Then, you have to run your Lambda function to publish messages. You can do that after deploying the code to AWS. Or, you can run it locally with a mockup of the Lambda env. Something like this:

const http = require('http');

// This is where the Lambda function is
const lambda = require('./lambda');

const context = {
    done: (error, success) => {
        if (error) {
            console.error('FAIL:', error);
            return;
        }
        console.log('OK:', success);
    },
};

const server = http.createServer((request, response) => {
    let data = '';
    request.on('data', (chunk) => {
        data += chunk;
    });
    request.on('end', () => {
        if (data) {
            const event = JSON.parse(data);
            lambda.handler(event, context);
        }
        response.end();
    });
});

server.on('clientError', (error, socket) => {
    socket.end('HTTP/1.1 400 Bad Request\r\n\r\n');
});

server.listen(3000);

This stub is a very basic mockup of Lambda env. It lacks some better error handling and validation. But for the purpose of this test, it does what we need. I strongly don’t recommend using this code in production, though.

If you named the script above, for instance, as web.js, you should be able to run it:

$ node web.js

And then invoke the function with cURL:

$ curl -d '{"message":"Hello world!"}' http://localhost:3000

The request body (with the -d param in the command) will be parsed as JSON and sent to the Lambda function as the event. If you check the function again, you’ll notice we’re using the message attribute there.

After executing that command, you should see two different outputs in your console. One from the Lambda mockup, which may look like this:

OK: { message: 'Hello world!' }

And another from the Artisan command:

Message received: Hello world!

The output will change in according to the message with the request body.

Conclusion

In this sample code, I showed the basics of Redis Pub/Sub. You don’t necessarily need AWS Lambda to use it. I just wanted to show up a “nearly” real-life use case. Sure, this is still not a real application, but I hope you got the idea.

You may have noticed, but this is a way to build what the cool kids out there call Microservices. If this is all new to you, maybe this is an opportunity to give it a chance and try to build your first distributed application.

Got comments or questions? Feel free to share them below.

Advertisements

Testing Stripe webhooks when using Laravel Cashier

I’m not sure why this is not in the docs, but if you’re using Laravel Cashier and want to test Stripe webhooks – in test, not live, mode – you have to set the following env var:

CASHIER_ENV=testing

I’ve spent some time checking around my code until I found that.

Injecting controller actions in Laravel views

Disclaimer: Depending on the kind of logic you need, it’s also possible to use View Composers to achieve a similar result.

I’m using Laravel in this new project I’m working on. Some other PHP frameworks have a feature to use controllers as services. Symfony, for instance, has something like that. The project team thought Laravel, as Symfony-based, would have something like that. Well, if it has, it’s not clear in the docs.

Another team member ended up with a solution I never thought before:

 @inject('someController', 'App\Http\Controllers\SomeController')
 {!! $someController->index() !!}

Now, we’re using Blade’s @inject directive to call controller actions from inside views. That’s useful for reusing actions as widgets, for example.

If you find that interesting and want to use in your application, remember two things:

  1. Since you’re calling the action method directly, you have to pass all the required params. If it expects a request instance, you can do this: $someController->index(request()).
  2. Probably the method returns a view that contains HTML code. So wrap the call within {!! and !!}. Using the {{ }} regular tag will cause the code to be escaped.

Implementing an SDK compatible with Laravel Container Service

When developing an SDK — and by SDK I mean an API implementation — you can leave for your users the task of integrating it with their apps. However, it’s a good idea to make it compatible with trendy frameworks out of the box.

In the PHP world, Laravel is becoming a very popular choice of framework. I want to share how you can make your SDK compatible with its Service Container.

The API client

For the sake of example, let’s imagine your API client class looks like this:

class Client
{
    public function __construct($username, $password)
    {
        // Set up client.
    }
}

Maybe you have a different way to set up API configuration. Anyway, it’s a good idea to have them set through a method, and not directly via a config file or other env-depending method. Using a method will make it easy to pass the settings from user’s app to the SDK. Also, it makes the configuration process more abstract and easier to plug-in.

The config file

In the sample client above, we have to pass a username and password to create a new instance. It’s clear we need to get that from somewhere. When using the SDK directly, you may do the following:

 define('API_USER', 'username');
 define('API_PASSWORD', '***');
 
 $client = new Client(API_USER, API_PASSWORD);

Since our goal is to inject the API implementation into the app, it’s better to config that using the framework way. Laravel stores application settings in different files inside a config dir. We can create a file like those to store the API settings:

<?php

return [

    // The API user.
    'username' => 'username',

    // The API password.
    'password' => '***',

];

Even better than set plain values to that config array, it’s using env vars. Laravel is shipped with PHP dotenv. It allows each environment to have its own settings without any change in the application code. So, let’s change our config file a little bit:

<?php

return [

    // The API user.
    'username' => env('API_USER'),

    // The API password.
    'password' => env('API_PASSWORD'),

];

The vars we’re using here have a very generic name. You should use a more specific name to avoid conflicts with other services. Something like DUMMY_API_USER, for an API called Dummy, for example.

The Service Provider

According to Laravel docs:

Service providers are the central place of all Laravel application bootstrapping. Your own application, as well as all of Laravel’s core services are bootstrapped via service providers.

But, what do we mean by “bootstrapped”? In general, we mean registering things, including registering service container bindings, event listeners, middleware, and even routes. Service providers are the central place to configure your application.

We have to create a service provider to tell Laravel that our API client can be injected as a dependency into application classes and methods. Also, the service provider will be in charge to merge the API configuration into the application configs.

It will look like this:

use Illuminate\Support\ServiceProvider;

class ApiServiceProvider extends ServiceProvider
{

    public function boot()
    {
        $this->publishes([
            __DIR__ . '/config.php' => config_path('api.php'),
        ]);
    }

    public function register()
    {
        $this->mergeConfigFrom(__DIR__ . '/config.php', 'api');

        $this->app->singleton('api.config', function ($app) {
            return $this->app['config']['api'];
        });

        $this->app->singleton(Client::class, function ($app) {
            $config = $app['api.config'];
            return new Client($config['username'], $config['password']);
        });
    }

    public function provides()
    {
        return [
            Client::class,
       ];
    }
}

In the boot method, we tell to Laravel which config files can be published to application’s config dir. So users of our API can overwrite those settings.

Within the register method, the service provider binds the config and the API client instance into the service container.

To improve performance, we use the provides method to let the framework know what are the binds this service provider offers. This way, it will only try to resolve the bind when it’s actually needed.

Using the service provider

After you added the SDK to the application, probably using Composer, you have to register its service provider. Open the config/app.php file of the app and add the service provider to the providers array:

$providers = [
    // ...

    ApiServiceProvider::class,
];

Now you can inject the API client into the app classes, like controllers:

class UserController extends Controller
{
    public function show(Client $client)
    {
        $user = $client->getUser();
        return view('user.show', [ 'user' => $user ]);
    }
}

To set the API username and password, you have to publish the config file:

$ php artisan vendor:publish --provider="ApiServiceProvider"

Then edit the config/api.php file if needed. This file may have another name if you changed its name in the service provider, which you should do.

You also may want to create the env vars inside the application’s .env.example and .env files.

Conclusion

Making your API SDK compatible with Laravel is very simple and requires only one extra class. It worth adding that to reach more users and make their work easier.

P.S.: I’ve omitted some implementation details and stuff like namespaces in the samples above. You can find a complete functioning example in Github: https://github.com/straube/dummy-sdk.

Laravel Logs to Sentry

If text-based logging is all you have, you should give a try to Sentry. It’s an amazing way to visualize and get notified about exceptions. And it supports a bunch of languages and frameworks. You can find more on its website, now I want to share a small tip to having your Laravel log outputted to Sentry.

When you set up Sentry to work with Laravel based on their guide, only exceptions will be sent to Sentry. If you have something like this:

try {
    throw new \Exception('Foo');
} catch (\Exception $e) {
    \Log::error($e);
}

That exception won’t reach the handler. So, Sentry won’t receive it.

There are a couple ways to forward that exception to Sentry, though. One of them is by doing the following:

try {
    throw new \Exception('Foo');
} catch (\Exception $e) {
    \Log::error($e);
    app('sentry')->captureException($e);
}

Yet, if you already wrote a lot of code, seeking for all your logging calls and adding that line can be a pain. To avoid that it’s possible to listen to the log event. Add a listener to your EventServiceProvider:

public function boot()
{
    parent::boot();
    Event::listen('illuminate.log', function ($level, $message, $context) {
        $sentry = app('sentry');
        if ($message instanceof \Exception) {
            $sentry->captureException($message, $context);
        } else {
            $sentry->captureMessage($message, null, $context);
        }
    });
}

This way all you’re logging calls will go to Sentry. You may found that some debug logging, for instance, may be pulled out from this logic. Add a condition to prevent it:

Event::listen('illuminate.log', function ($level, $message, $context) {
    if (in_array($level, [ 'debug', 'info' ])) {
        return;
    }

    // ...

});

BTW, Happy new year! ;)