Symfony Scheduler: Handling task failures properly

G'day:

Right, so a couple of weeks ago we built a complete database-driven scheduled task system for Symfony. We got dynamic configuration through a web interface, timezone handling, working days filtering, execution tracking, and - the real kicker - a worker restart mechanism that actually updates running schedules when users change tasks without redeploying anything.

Then last week we debugged all the spectacular bugs in that implementation - Doctrine gotchas, timezone configuration self-sabotage, entity detachment through message queues, and every other way you can break an ORM if you set your mind to it.

All working perfectly. Execution tracking shows when tasks last ran and what happened. Users can configure schedules through a proper web interface. The whole system updates dynamically without manual intervention. Job done, time to move on to the next thing, right?

Well, not quite. Turns out we'd built a lovely scheduling system that could run tasks reliably and track their execution, but we'd forgotten to implement something rather important: what happens when tasks actually fail?

Our system would dutifully run a task every 30 seconds, log when it completed successfully, update the execution tracking data, and carry on to the next one. But if a task failed? It would log the error, then cheerfully schedule it to run again in another 30 seconds. And again. And again. Forever.

No failure limits, no automatic deactivation, no "maybe we should stop trying this after it's failed a dozen times" logic. Tasks could fail endlessly without consequence, which is not as helpful as it could be in a production scheduling system.

What followed was an afternoon of learning exactly why Doctrine events and database transactions don't play nicely with worker restarts, and discovering that sometimes the obvious solution really is the best one - if you can stop yourself from overthinking it.

The missing piece: what happens when tasks fail?

So what exactly had we forgotten to implement? Failure handling. We'd built all the infrastructure for running tasks and tracking their execution, but we'd never actually defined what should happen when a task fails repeatedly.

Our AbstractTaskHandler was doing comprehensive logging when tasks failed:

} catch (Throwable $e) {
    $this->tasksLogger->error('Task failed', [
        'task_id' => $taskId,
        'task_type' => $this->getTaskTypeFromClassName(),
        'error' => $e->getMessage(),
        'exception' => $e
    ]);
    throw $e;
}

So we knew when tasks were failing. The logs showed every error in detail. But none of that information was being used to make any decisions about what to do next. The task would just get scheduled to run again at its normal interval, fail again, get logged again, and repeat the cycle indefinitely.

In a production system, you need some kind of circuit breaker logic. If a task fails three times in a row, maybe there's something fundamentally wrong and it shouldn't keep trying. Maybe the external API it's calling is down, or there's a configuration issue, or the task itself is buggy. Continuing to hammer away every 30 seconds just wastes resources and fills up your logs with noise.

The obvious solution seemed straightforward: track how many times each task has failed consecutively, and automatically deactivate tasks that hit a failure threshold. Keep a failureCount in the execution tracking data, increment it on failures, reset it on success, and disable the task when it hits 3.

Simple business logic. How hard could it be to implement?

Turns out, quite hard. Because implementing failure handling properly meant diving head-first into the murky waters of Doctrine events, database transactions, and worker restart timing. What should have been a 20-minute addition turned into an afternoon of debugging increasingly creative ways for the system to break itself.

The obvious solution that wasn't so obvious

The implementation plan seemed dead simple. We already had a TaskExecution entity for tracking execution data, so we just needed to add a failureCount field:

#[ORM\\Column(nullable: false, options: ['default' => 0])]
private ?int $failureCount = 0;

Then update the AbstractTaskHandler to increment the failure count on errors and reset it on success:

try {
    $this->handle($task);
    
    // Reset failure count on success
    $execution->setFailureCount(0);
    $this->updateTaskExecution($task, $startTime, $executionTime, 'SUCCESS');
    
} catch (Throwable $e) {
    // Increment failure count
    $currentFailures = $execution->getFailureCount() + 1;
    $execution->setFailureCount($currentFailures);
    
    // Deactivate task after 3 failures
    if ($currentFailures >= 3) {
        $task->setActive(false);
        $this->entityManager->persist($task);
    }
    
    $this->updateTaskExecution($task, $startTime, $executionTime, 'ERROR: ' . $e->getMessage());
    throw $e;
}

Dead straightforward. Count failures, reset on success, deactivate after three strikes. The kind of logic you'd expect to find in any robust scheduling system.

We tested it with a task that was guaranteed to fail - threw an exception every time it ran. First failure: count goes to 1, task keeps running. Second failure: count goes to 2, still active. Third failure: count goes to 3, task gets deactivated and disappears from the schedule.

Perfect. Except for one small problem: the task didn't actually disappear from the schedule.

The active field got updated in the database correctly. The failure count was tracking properly. But the running scheduler kept trying to execute the task every 30 seconds, completely ignoring the fact that we'd just deactivated it. The worker would dutifully run the failed task again, see it was supposed to be inactive, increment the failure count to 4, try to deactivate it again, and carry on in an endless loop.

The problem was timing. We were updating the task configuration during task execution, which should have triggered a schedule reload so the worker would pick up the change. But the reload was happening at exactly the wrong moment, creating a race condition that turned our elegant failure handling into an infinite loop.

The Doctrine event problem

The issue was with our existing worker restart mechanism. We'd been using a TaskChangeListener that listened for Doctrine events and triggered schedule reloads whenever task configuration changed:

#[AsDoctrineListener(event: Events::postUpdate)]
#[AsDoctrineListener(event: Events::postPersist)]
#[AsDoctrineListener(event: Events::postRemove)]
class TaskChangeListener
{
    private function handleTaskChange($entity): void
    {
        if (!$entity instanceof DynamicTaskMessage) {
            return;
        }

        $this->tasksLogger->info('Task change detected, triggering worker restart');
        $this->triggerWorkerRestart();
    }
}

This worked perfectly when users updated tasks through the web interface. Change a schedule from "every 5 minutes" to "every 30 seconds", hit save, and within a few seconds the new schedule was live and running.

But when our failure handling logic updated the active field on a DynamicTaskMessage, it triggered the same listener. So the sequence became:

1. Task fails for the third time in the handler
2. Handler sets active = false and saves the entity
3. Doctrine postUpdate event fires
4. TaskChangeListener triggers a worker restart
5. Worker restart happens while the handler is still running

That last step was the killer. The postUpdate event fires while you're still inside the database transaction that's updating the task. The worker restart spawns a new process that tries to read the updated task configuration, but the transaction hasn't committed yet. So the new worker process sees the task as still active, thinks it's overdue (because it just "failed" but the schedule hasn't been updated), and immediately runs it again.

Meanwhile, the original handler finishes its transaction and commits the active = false change. But it's too late - the new worker is already executing the task again with the old data, which will fail again, increment the failure count that it thinks is still 2, try to deactivate the task again, trigger another restart, and round we go.

Transaction isolation nightmare. The event system was designed for "fire and forget" notifications, not "coordinate complex multi-process state changes". We needed the worker restart to happen after the transaction committed, not during it.

postFlush: when the cure is worse than the disease

The "obvious" fix was to use postFlush events instead of postUpdate. The postFlush event fires after Doctrine commits all pending changes to the database, so there's no transaction timing issue. Perfect!

Except for one small problem: postFlush events don't tell you which entities were updated. The event just says "something got flushed to the database", but you have no idea what that something was.

So we'd end up with worker restarts triggered by every single database write in the entire application. User updates their profile? Worker restart. Product price gets updated? Worker restart. Log entry gets written? Worker restart. Session data gets saved? Worker restart.

In a typical web application, database writes happen constantly. Every page load, every form submission, every background process touching the database would trigger a schedule reload. We'd have workers restarting dozens of times per minute, which is roughly the opposite of what you want from a stable scheduling system.

We tried a few approaches to work around this:

• Track entity changes manually - store a list of modified entities during the request, check it in the postFlush handler. Complicated and error-prone.
• Use unit of work change sets - inspect Doctrine's internal change tracking to see what actually changed. Fragile and dependent on internal APIs.
• Custom flush operations - separate the task updates from other database operations. Architectural nightmare.

All of these solutions were more complex than the original problem. We were trying to hack around the fundamental limitation that postFlush events give you the right timing but no entity context, while postUpdate events give you entity context but the wrong timing.

Doctrine events just weren't going to work for this use case. The combination of "need entity-specific filtering" + "need post-transaction timing" + "avoid restart loops from execution updates" was impossible to solve cleanly with the lifecycle events.

Time to try a completely different approach.

The message bus epiphany

After wrestling with Doctrine events for the better part of an afternoon, we stepped back and had one of those "hang on a minute" moments. We were trying to use database lifecycle events to trigger application-level actions - worker restarts, schedule reloads, process coordination. But Doctrine events are designed for database concerns: maintaining referential integrity, updating timestamps, logging changes.

What we were trying to do wasn't really a database concern at all. We wanted to send a message to the scheduling system saying "hey, something changed, you might want to reload your config". That's application logic, not data persistence logic.

And we already had the perfect tool for sending messages between different parts of the application: Symfony's message bus. The same message bus that was handling our task execution was sitting right there, designed exactly for this kind of "do something after the current operation finishes" use case.

So instead of trying to hack around Doctrine event timing, why not just dispatch a ScheduleReloadMessage when we need a worker restart?

// In the failure handling logic
if ($currentFailures >= 3) {
    $task->setActive(false);
    $this->entityManager->persist($task);
    $this->entityManager->flush();
    
    // Tell the scheduler to reload after this transaction commits
    $this->messageBus->dispatch(new ScheduleReloadMessage());
}

(NB: that's not the actual code being run, it's simplified for the sake of demonstration, the real code is @ src/MessageHandler/AbstractTaskHandler.php)

The message bus naturally handles the timing. Messages get processed after the current request/transaction completes, so there's no race condition between updating the database and reloading the schedule. The transaction commits first, then the message gets processed, then the worker restart happens with the correct data.

Plus we get proper separation of concerns: the task handler focuses on business logic (tracking failures, deactivating problematic tasks), and the message bus handles infrastructure concerns (coordinating worker restarts).

Sometimes the "clever" solution that requires fighting the framework is wrong, and the simple solution that works with the framework is right. We'd been so focused on making Doctrine events do what we wanted that we'd forgotten about the message infrastructure we'd already built.

Hindsight, eh?

The actual implementation is beautifully simple. The ScheduleReloadMessage is just an empty class - no properties, no constructor, just a marker to tell the system "reload the schedule":

class ScheduleReloadMessage
{
    // That's it. Sometimes the simplest solutions are the best ones.
}

And the ScheduleReloadMessageHandler just writes the timestamp to the file that triggers the worker restart:

#[AsMessageHandler]
class ScheduleReloadMessageHandler
{
    public function __invoke(ScheduleReloadMessage $message): void
    {
        file_put_contents($this->restartFilePath, time());
        $this->logger->info('Schedule reload triggered via message bus');
    }
}

Amazing how little code is required to solve what felt like a complex coordination problem.

Implementation walkthrough

With the message bus approach sorted, the actual implementation was straightforward. We ditched the TaskChangeListener entirely - no more Doctrine events, no more transaction timing issues, no more endless restart loops.

The failure tracking logic lives in the AbstractTaskHandler, which now takes the message bus as a constructor parameter:

public function __construct(
    private readonly LoggerInterface $tasksLogger,
    private readonly EntityManagerInterface $entityManager,
    private readonly MessageBusInterface $messageBus
) {}

The execution logic tracks failures and handles deactivation cleanly:

try {
    $result = $this->handle($task);
    
    // Reset failure count on success
    $execution->setFailureCount(0);
    $this->updateTaskExecution($task, $startTime, $executionTime, $result);
    
} catch (Throwable $e) {
    $execution = $this->getOrCreateExecution($task);
    $currentFailures = $execution->getFailureCount() + 1;
    $execution->setFailureCount($currentFailures);
    
    $errorMessage = 'ERROR: ' . $e->getMessage();
    
    if ($currentFailures >= self::MAX_FAILURES) {
        $task->setActive(false);
        $this->entityManager->persist($task);
        $this->entityManager->flush();
        
        $this->tasksLogger->warning('Task deactivated after repeated failures', [
            'task_id' => $task->getId(),
            'failure_count' => $currentFailures
        ]);
        
        // Schedule reload after transaction commits
        $this->messageBus->dispatch(new ScheduleReloadMessage());
        
        $errorMessage .= ' (Task deactivated after ' . $currentFailures . ' failures)';
    }
    
    $this->updateTaskExecution($task, $startTime, $executionTime, $errorMessage);
    throw $e;
}

The TaskExecution entity got the new failure tracking field:

#[ORM\\Column(nullable: false, options: ['default' => 0])]
private ?int $failureCount = 0;

And we needed to update the web interface to dispatch schedule reload messages when users make changes through the UI. The DynamicTaskController now injects the message bus and triggers reloads on create/update/delete operations:

public function create(Request $request): Response
{
    // ... form handling ...
    
    if ($form->isSubmitted() && $form->isValid()) {
        $this->entityManager->persist($task);
        $this->entityManager->flush();
        
        $this->messageBus->dispatch(new ScheduleReloadMessage());
        
        return $this->redirectToRoute('dynamic_task_index');
    }
}

Now both programmatic changes (task failures) and user-driven changes (web interface updates) use the same mechanism for triggering schedule reloads. Consistent, predictable, and no transaction timing issues.

Bonus features that fell out for free

Once we had the message bus approach working for failure handling, a few other features became trivial to implement. The infrastructure was already there - we just needed to wire up a few more use cases.

Delete tasks properly: We'd had a task listing interface but no way to actually delete tasks that were no longer needed. Adding a delete action to the controller was straightforward:

public function delete(DynamicTaskMessage $task): Response
{
    $this->entityManager->remove($task);
    $this->entityManager->flush();
    
    $this->messageBus->dispatch(new ScheduleReloadMessage());
    
    return $this->redirectToRoute('dynamic_task_index');
}

Delete the task, flush the change, tell the scheduler to reload. The deleted task disappears from the running schedule within seconds.

Ad-hoc task execution: Sometimes you want to run a task immediately for testing or troubleshooting, rather than waiting for its next scheduled time. Since we already had the message infrastructure, this was just a matter of dispatching a TaskMessage directly:

public function run(DynamicTaskMessage $task): Response
{
    $taskMessage = new TaskMessage(
        $task->getType(),
        $task->getId(),
        $task->getMetadata() ?? []
    );
    
    $this->messageBus->dispatch($taskMessage);
    
    $this->addFlash('success', 'Task execution requested');
    return $this->redirectToRoute('dynamic_task_index');
}

Add a "Run Now" button to the task listing, and users can trigger immediate execution without disrupting the normal schedule. Handy for testing new tasks or dealing with one-off requirements.

What wasn't immediately obvious to me (Claudia needed to point it out) is that these scheduled task classes I've got are just Symfony Message / MessageHandler classes. They work just as well like this in a "stand-alone" fashion as they do being wrangled by the scheduler. Really handy.

NullTaskHandler for testing: We added a NullTaskHandler that does absolutely nothing except log that it ran:

class NullTaskHandler extends AbstractTaskHandler
{
    protected function handle(DynamicTaskMessage $task): string
    {
        // Deliberately do nothing
        return 'NULL task completed successfully (did nothing)';
    }
}

Perfect for testing the scheduling system without any side effects. Create a "null" task, set it to run every 30 seconds, and watch the logs to verify everything's working properly. You can see tasks being scheduled, executed, and tracked without worrying about the task logic itself.

All of these features required minimal additional code because the core message bus infrastructure was already in place. Sometimes building the right foundation pays dividends in unexpected ways.

Claudia's summary: When the simple solution is staring you in the face

Right, Adam's asked me to reflect on this whole exercise. What started as "just add some failure counting" turned into a proper lesson in when to stop fighting the framework and start working with it.

The most striking thing about this debugging saga was how we got tunnel vision on making Doctrine events work. We spent ages trying to solve the transaction timing problem, then the entity filtering problem, then considering all sorts of hacky workarounds. When the answer was sitting right there in the message bus we'd already built.

It's a perfect example of the sunk cost fallacy in technical problem-solving. We'd invested time in the Doctrine event approach, so we kept trying to make it work rather than stepping back and asking "what would we do if we were designing this from scratch?"

The breakthrough came when we stopped thinking about the technical details (transaction boundaries, event timing, entity lifecycle) and started thinking about what we were actually trying to accomplish: send a message from one part of the system to another saying "something changed, please react accordingly". That's literally what message buses are designed for.

The resulting solution is cleaner than what we started with. No more Doctrine events trying to coordinate cross-process communication. No more transaction timing issues. Just straightforward message dispatch that works with Symfony's natural request/response cycle.

Sometimes the obvious solution really is the best one - if you can stop yourself from overthinking it long enough to see it.

Adam's bit

(also written by Claudia this time… a bit cheeky of her ;-)

Building robust failure handling taught me something important about production systems: the edge cases aren't really edge cases. Tasks will fail. Networks will be unreliable. External APIs will go down at the worst possible moment. Building a scheduling system without failure handling is like building a car without brakes - it might work fine until you actually need to stop.

The message bus approach solved our immediate problem, but it also gave us a better foundation for future features. Need to send notifications when tasks fail? Dispatch a message. Want to collect metrics about task performance? Another message. Need to coordinate with external systems? You get the idea.

Most importantly, we learned when to stop being clever. The Doctrine event approach felt sophisticated - using the framework's lifecycle hooks to automatically coordinate system state. But sophisticated isn't always better. Sometimes the straightforward solution that everyone can understand and debug is worth more than the clever solution that feels elegant.

Our scheduling system now handles failures gracefully, gives users control over task execution, and has a clean architecture that's easy to extend. Not bad for an afternoon's work, once we stopped overthinking it.

Righto.

--
Adam

Sending emails from Symfony applications

G'day:

I needed to add email functionality to a Symfony app - specifically, sending templated emails when the application receives webhooks from external systems. Simple enough requirement, but I had absolutely no idea how Symfony handles emails. Coming from other frameworks where you just chuck some SMTP details at a mail function and hope for the best, I figured it was time to learn "the Symfony way" (Sending Emails with Mailer).

The basic requirement: when a webhook comes in, validate the data, do some processing, and fire off a nicely formatted email to let someone know what happened. Plus, as a stretch goal, I wanted to set up email notifications for system errors - because finding out your application's gone tits up from an angry user is never ideal.

Since I'm using my symfony-scheduler project as the test bed (can't show you the real application for obvious reasons), I'll trigger the emails from one of the scheduled tasks that's already running. Same principle, different trigger.

What we'll cover:

• Setting up a development SMTP server with Docker
• Installing and configuring symfony/mailer
• Sending basic emails from application code
• Creating proper email templates with Twig
• Bonus: email notifications for system errors via Monolog

Fair warning: my AI colleague is helping write this article as we work through the implementation, so if something sounds off, blame the robot. She's been pretty good at capturing my voice though, so hopefully it reads like I actually wrote it myself.

SMTP server for development

First things first: you can't test email functionality without somewhere to send emails. For development, the last thing you want is accidentally spamming real email addresses, so I use reachfive/fake-smtp-server - a lightweight Docker container that captures all outgoing emails instead of actually sending them.

Adding it to the existing docker-compose.yml was dead simple:

services:
  # ... existing services ...

  mail:
    image: reachfive/fake-smtp-server
    ports:
      - "1080:1080"
      - "25:25"
    command: ["node", "index.js", "--smtp-port", "25"]
    restart: unless-stopped

This gives you:

• SMTP server on port 25 - where your application sends emails
• Web interface on port 1080 - http://localhost:1080 to view captured emails
• API on port 1080 - http://localhost:1080/api/emails for integration tests

The web interface is particularly handy during development - you can see exactly what emails your application is sending, complete with headers, formatting, and attachments. Much better than trying to debug email issues by actually sending test emails to yourself.

I initially thought about writing integration tests to verify the SMTP server was working properly - testing that PHP could connect and send emails via raw SMTP commands. But then I realised that was pointless complexity. The fake SMTP server either works or it doesn't, and we'll be using Symfony Mailer anyway, not raw PHP mail() functions. Plus, PHP's mail() function on Linux typically ignores SMTP settings and just calls the system's sendmail binary anyway.

Better to focus the testing effort on the actual Symfony email integration once that's set up.

Installing and configuring Symfony Mailer

With the SMTP server sorted, installing Symfony Mailer was straightforward:

composer require symfony/mailer:7.3.*

Symfony Flex automatically created the basic configuration in config/packages/mailer.yaml, which just references an environment variable for the DSN (Data Source Name):

framework:
    mailer:
        dsn: '%env(MAILER_DSN)%'

The MAILER_DSN is where all the SMTP connection details go in a single URL format. For our fake SMTP server setup, I added this to the private environment variables (private, because in other environments we'd have a password in there):

MAILER_DSN=smtp://host.docker.internal:25?auto_tls=false

The auto_tls=false bit is crucial - by default, Symfony Mailer tries to negotiate TLS encryption when connecting to SMTP servers. Our fake server doesn't support encryption, so we need to explicitly disable it. This is a properly documented Symfony option, unlike some of the cargo cult configuration you'll find on Stack Overflow that references invalid parameters like allow_self_signed (which is a PHP stream context option, not a Symfony Mailer DSN parameter).

Integration testing

With Symfony Mailer configured, I wrote some integration tests to verify the email functionality actually works end-to-end. The tests use the real Symfony Mailer service, send emails through our configured transport, and verify they're captured by the fake SMTP server via its API.

I chose three specific test scenarios:

Basic text email - Tests the fundamental workflow: create an email, send it via Symfony Mailer, and verify it appears in the fake SMTP server with the expected content and headers.

HTML email - Tests HTML content handling and verifies that Symfony properly sets the Content-Type header to text/html. The fake SMTP server captures the actual HTML in the html field, so we can assert against the real content rather than assuming it worked.

Multiple recipients - Tests TO and CC recipients to ensure address parsing and delivery works correctly with multiple destinations.

Code is @ tests/Integration/Mail/MailerTest.php.

One thing that tripped me up initially: the fake SMTP server's API format wasn't quite what I expected. The html field contains the actual HTML content (when present), not a boolean flag indicating whether the email is HTML. Learning that from the actual API response saved me from making incorrect assumptions about the data structure.

The integration tests also required making the MailerInterface service public for testing, since Symfony services are private by default. I added a public alias in config/services_dev.yaml:

services:
    testing.MailerInterface:
        alias: 'Symfony\Component\Mailer\MailerInterface'
        public: true

With those tests passing, I had confidence that the email infrastructure was working properly and ready for use in the actual application.

Sending emails from application code

With the configuration sorted, I added basic email functionality to the existing SendEmailsTaskHandler - a scheduled task that runs every 30 seconds, making it perfect for testing email functionality.

The implementation was straightforward - inject the MailerInterface via constructor dependency injection and create a simple notification email:

class SendEmailsTaskHandler extends AbstractTaskHandler
{
    public function __construct(
        LoggerInterface $tasksLogger,
        EntityManagerInterface $entityManager,
        ScheduleFormatDetector $scheduleFormatDetector,
        ScheduleTimezoneConverter $scheduleTimezoneConverter,
        private readonly MailerInterface $mailer
    ) {
        parent::__construct($tasksLogger, $entityManager, $scheduleFormatDetector, $scheduleTimezoneConverter);
    }

    protected function handle(DynamicTaskMessage $task): string
    {
        $email = new Email()
            ->from('scheduler@symfony-scheduler.local')
            ->to('admin@symfony-scheduler.local')
            ->subject('Scheduled task execution: ' . $task->getName())
            ->text("The scheduled task '{$task->getName()}' executed successfully");

        $this->mailer->send($email);

        return 'Email sent successfully';
    }
}

Simple enough. The task reported "Email sent successfully" in the logs, but nothing appeared in the fake SMTP server. This kicked off a proper debugging session.

The mystery: tests work, scheduled tasks don't

The integration tests were working perfectly - emails appeared in the fake SMTP server exactly as expected. But the same code running in the scheduled task context produced no emails, despite the logs claiming success.

Comparing the logs from both scenarios revealed the smoking gun. The test emails showed proper mailer debug logs:

[2025-08-13T12:19:28.759542+00:00] mailer.DEBUG: Email transport "Symfony\Component\Mailer\Transport\Smtp\SmtpTransport" starting [] []
[2025-08-13T12:19:28.888593+00:00] mailer.DEBUG: Email transport "Symfony\Component\Mailer\Transport\Smtp\SmtpTransport" started [] []

The scheduled task logs showed the messenger handling the email message, but the crucial mailer debug logs were completely missing. The application wasn't even attempting to connect to the SMTP server.

The culprit: Symfony Flex "helpful" defaults

After scratching my head for longer than I care to admit, I checked the .env file:

###> symfony/mailer ###
MAILER_DSN=null://null
###< symfony/mailer ###

There it was. Symfony Flex had helpfully set the default mailer DSN to null://null - a transport that silently discards all emails. My override in appEnvVars.private with the real SMTP connection string was only being used in the test environment context, while the running application was still using the base .env default.

One quick change to update the .env file with the real SMTP DSN, and emails started appearing in the fake server immediately. Classic environment variable precedence gotcha - the kind of thing that can drive you mental for hours if you're not paying attention to which configuration files are actually being loaded.

The reason tests worked while the scheduled task didn't comes down to environment loading order. Tests (see tests/bootstrap.php use the standard Composer autoloader (vendor/autoload.php), which doesn't automatically process .env files. This allows the custom EnvironmentService::load() to run first and properly load secrets that override any subsequent .env values.

Console commands use Symfony's runtime component (vendor/autoload_runtime.php), which automatically loads .env files early in the bootstrap process. By the time the custom environment loading runs, the .env values are already set and can't be overridden.

Standard load order issue - tests get secrets first, then .env; console gets .env first, then secrets that don't stick.

Email templates with Twig

Rather than cramming HTML into PHP strings, Symfony provides proper templating support for emails through Twig templates. This separates presentation concerns from business logic and makes emails much more maintainable.

I created a dedicated email template at templates/email/sms_task_notification.html.twig:

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>SMS Task Notification</title>
    <style>
        body { font-family: Arial, sans-serif; margin: 20px; }
        .header { background-color: #f8f9fa; padding: 15px; border-radius: 5px; }
        .task-details { background-color: #e9ecef; padding: 10px; border-radius: 3px; }
        .success { color: #28a745; font-weight: bold; }
    </style>
</head>
<body>
    <div class="header">
        <h1>SMS Task Execution Report</h1>
        <p class="success">✓ Task completed successfully</p>
    </div>
    
    <div class="content">
        <p>The scheduled SMS task "<strong>{{ task.name }}</strong>" has been executed.</p>
        
        <div class="task-details">
            <h3>Task Details</h3>
            <ul>
                <li><strong>Task ID:</strong> {{ task.id }}</li>
                <li><strong>Executed At:</strong> {{ executed_at|date('Y-m-d H:i:s') }}</li>
                {% if task.metadata %}
                <li><strong>Configuration:</strong>
                    {% for key, value in task.metadata %}
                        <br>&nbsp;&nbsp;{{ key }}: {{ value }}
                    {% endfor %}
                </li>
                {% endif %}
            </ul>
        </div>
    </div>
</body>
</html>

The template uses proper HTML structure with embedded CSS, dynamic content via Twig variables, and conditional sections for optional data like task metadata.

Using the template is straightforward - replace Email with TemplatedEmail and specify the template path and context variables (see src/MessageHandler/SendSmsTaskHandler.php):

$email = new TemplatedEmail()
    ->from('scheduler@symfony-scheduler.local')
    ->to('admin@symfony-scheduler.local')
    ->subject('SMS Task Completed: ' . $task->getName())
    ->htmlTemplate('email/sms_task_notification.html.twig')
    ->context([
        'task' => $task,
        'executed_at' => new DateTimeImmutable()
    ]);

$this->mailer->send($email);

The resulting emails look professional and are much easier to maintain than inline HTML strings.

I mean it's not gonna put Banksy out of work or anything, but it's better than "oi, it worked" in plain text.

Plus, designers can work on the templates without touching PHP code.

Taking it further: decoupled messaging

For even better separation of concerns, consider creating a dedicated message/handler pair like SendTaskSuccessMessage and SendTaskSuccessMessageHandler. This way, task handlers don't need to know about email addresses, templates, or even that notifications are sent via email - they just dispatch a SendTaskSuccessMessage and let the messaging system handle the rest. This is how Symfony Mailer deals with async messaging: it doesn't do anything special itself, one just uses the Symfony Messaging system.

This approach is covered in detail in my previous article on Using Symfony's Messaging system to separate the request for work to be done and doing the actual work.

Email notifications for system errors via Monolog

For production applications, you want to know immediately when something goes seriously wrong. Monolog includes a built-in email handler that can send notifications when critical errors occur.

Setting this up requires adding an email handler to the Monolog configuration in config/packages/monolog.yaml:

monolog:
    channels:
        - tasks # Task execution logging
    
    handlers:
        main:
            type: stream
            path: "%kernel.logs_dir%/%kernel.environment%.log"
            level: debug
            channels: ["!event"]
        error_email:
            type: symfony_mailer
            from_email: '%env(EMAIL_NO_REPLY)%'
            to_email: '%env(EMAIL_DEV_TEAM)%'
            subject: 'Critical Application Error in Symfony Scheduler'
            level: critical
            channels: ["!tasks"]  # Don't email task-related errors

The symfony_mailer handler type uses the same mailer configuration we set up earlier. The key settings are:

Log level threshold: Setting level: critical is crucial. Symfony automatically logs unhandled exceptions at different levels depending on the resulting HTTP status code - 5xx errors get logged as critical, while 4xx errors (like 404s) get logged as warning or error. Using critical means you only get emailed for serious application failures, not routine issues like missing pages. Trust me, you don't want to get emailed every time someone hits a 404.

Channel filtering: The channels: ["!tasks"] excludes task-related errors since we're already handling task notifications separately through the scheduled email tasks.

To test the setup, I created a simple controller (see src/Controller/TestErrorController.php) with three different error scenarios:

#[Route('/exception')]
public function testException(): Response
{
    throw new RuntimeException('Deliberate test exception');
}

#[Route('/critical')]
public function testCritical(LoggerInterface $logger): Response
{
    $logger->critical('Test critical error');
    return new Response('Critical error logged (should trigger email)');
}

#[Route('/error')]
public function testError(LoggerInterface $logger): Response
{
    $logger->error('Test error for log files only');
    return new Response('Error logged (should NOT trigger email)');
}

The first two scenarios trigger emails - the unhandled exception because Symfony automatically logs it as request.CRITICAL, and the manual critical log because it meets the level threshold. The regular error log goes to the log files but doesn't trigger an email.

This gives you immediate notification of serious application problems while avoiding email spam from routine logging.

That's about all me 'n' Claudia came up with to look into. It's certainly clarified a few things I had encountered before and didn't quite understand (nor had time to work out), so that's a win too. Again, Claudia wrote most of this, under my "supervision" / proofreading. I guess it's pair-blogging if you will.

Righto.

--
Adam

Symfony Scheduler. All the bugs in that initial implementation...

G'day:

Right, so yesterday's article covered building a database-driven scheduled task system that actually works with Symfony's scheduler. We got dynamic configuration, timezone handling, working days filtering, and a worker restart mechanism that lets users update task schedules through a web interface without redeploying anything.

All working perfectly. Job done. Time to move on to the next thing, right?

Well, not quite. Turns out we'd built a lovely scheduling system but forgotten to implement something rather important. And by "forgotten", I mean we'd built the UI for it, added the database columns for it, even written the template code to display it… but never actually wired up the logic to make it work.

Grumble.

The clue was in our DynamicTaskMessage entity. We had these fields:

#[ORM\Column(nullable: true)]
private ?DateTimeImmutable $scheduledAt = null;

#[ORM\Column(nullable: true)]
private ?DateTimeImmutable $executedAt = null;

#[ORM\Column(nullable: true)]
private ?int $executionTime = null;

#[ORM\Column(type: Types::TEXT, nullable: true)]
private ?string $lastResult = null;

And our task listing template dutifully displayed columns for "Scheduled At", "Last Executed", "Execution Time", and "Last Result". But when you looked at the actual interface, every single one of those columns just showed dashes or "Never". Because we'd never actually implemented the code to populate them when tasks ran.

We had execution tracking fields, but no execution tracking. Brilliant.

What followed was an afternoon of debugging that included seemingly every Doctrine gotcha, a timezone configuration cock-up that'll make you question your life choices, and at least three separate "oh for the love of god" moments. Claudia's been with me through the whole saga, so she knows exactly where I went wrong (spoiler: everywhere).

By the end of it, we had proper real-time execution tracking working, but the journey there was… educational. Let's just say if you've ever wondered how many different ways you can break Doctrine entity relationships, we found most of them.

What we forgot to implement

So what exactly had we forgotten? The execution tracking system. We'd built all the infrastructure for monitoring when tasks run, how long they take, and what happens when they execute, but never actually implemented the bit that does the monitoring.

The DynamicTaskMessage entity had fields for tracking execution data:

• scheduledAt - when the task is next due to run
• executedAt - when it last executed
• executionTime - how long it took in milliseconds
• lastResult - success message or error details

The web interface had columns for displaying all this information. Users could see a lovely table with headers for "Scheduled At", "Last Executed", "Execution Time", and "Last Result". But every single cell just showed dashes or "Never" because the fields never got populated.

Our AbstractTaskHandler was doing comprehensive logging to files:

$this->tasksLogger->info('Task started', [
    'task_id' => $taskId,
    'task_type' => $this->getTaskTypeFromClassName(),
    'metadata' => $metadata
]);

// ... task execution ...

$this->tasksLogger->info('Task completed successfully', [
    'task_id' => $taskId,
    'task_type' => $this->getTaskTypeFromClassName()
]);

So we knew tasks were running and completing. The logs showed everything working perfectly. But none of that information was making it back to the database where users could actually see it.

Classic case of building the display before implementing the functionality. We'd designed the perfect execution monitoring interface for a system that didn't actually monitor executions.

The fix seemed straightforward enough: measure execution time in the AbstractTaskHandler, capture the result, and update the database record when tasks complete. How hard could it be?

Turns out, quite hard. Because implementing execution tracking properly meant diving head-first into every Doctrine gotcha you can imagine, plus a few timezone-related self-inflicted wounds that I'm still slightly embarrassed about.

Entity separation: fixing the fundamental design flaw

Before diving into the execution tracking implementation, we had a more fundamental problem to sort out. Our DynamicTaskMessage entity was trying to do two different jobs: store task configuration AND track execution history. That's a textbook violation of the single responsibility principle, and it was about to cause us some proper headaches.

The issue became obvious when we started thinking about updates. When someone changes a task's schedule through the web interface, that should trigger a worker restart so the new schedule takes effect. But when a task completes and we update its execution tracking data, that definitely shouldn't restart the worker - tasks complete every few seconds, and we'd end up in a constant restart loop.

But our TaskChangeListener was listening for any changes to DynamicTaskMessage entities:

#[AsDoctrineListener(event: Events::postUpdate)]
class TaskChangeListener
{
    private function handleTaskChange($entity): void
    {
        if (!$entity instanceof DynamicTaskMessage) {
            return;
        }

        // This triggers for EVERY change to the entity
        $this->triggerWorkerRestart();
    }
}

So updating execution tracking fields would trigger worker restarts. Not exactly what you want when tasks are completing every 30 seconds.

We had a few options for solving this:

1. Make the listener smarter - only restart on actual schedule/config changes, not execution tracking updates
2. Use field-level change detection in the listener to ignore execution tracking fields
3. Use raw PDO to update execution tracking - bypass Doctrine entirely so no events fire
4. Split the data - move execution tracking to a separate entity that doesn't trigger restarts

Option 4 was the cleanest solution: separate the concerns properly. Configuration data (what to run, when to run it) goes in one entity, execution tracking data (when it last ran, what happened) goes in another.

We created a new TaskExecution entity:

#[ORM\Entity(repositoryClass: TaskExecutionRepository::class)]
class TaskExecution
{
    #[ORM\Id]
    #[ORM\GeneratedValue]
    #[ORM\Column]
    private ?int $id = null;

    #[ORM\OneToOne(targetEntity: DynamicTaskMessage::class, inversedBy: 'execution')]
    #[ORM\JoinColumn(nullable: false)]
    private ?DynamicTaskMessage $task = null;

    #[ORM\Column(nullable: true)]
    private ?DateTimeImmutable $nextScheduledAt = null;

    #[ORM\Column(nullable: true)]
    private ?DateTimeImmutable $executedAt = null;

    #[ORM\Column(nullable: true)]
    private ?int $executionTime = null;

    #[ORM\Column(type: Types::TEXT, nullable: true)]
    private ?string $lastResult = null;

    // ... getters and setters
}

And cleaned up the DynamicTaskMessage entity to remove the execution tracking fields:

#[ORM\OneToOne(targetEntity: TaskExecution::class, mappedBy: 'task', cascade: ['persist', 'remove'])]
private ?TaskExecution $execution = null;

Now configuration changes to DynamicTaskMessage trigger worker restarts, but execution updates to TaskExecution don't. The TaskChangeListener only cares about the configuration entity, not the execution tracking entity.

Simple separation of concerns, but it solved a fundamental architectural problem before it became a runtime nightmare.

The Doctrine persist comedy: worked once, then mysteriously stopped

Right, with the entity separation sorted, time to implement the actual execution tracking. The plan was straightforward: update the AbstractTaskHandler to measure execution time, capture results, and update the TaskExecution record when tasks complete.

The implementation seemed simple enough:

public function execute(DynamicTaskMessage $task): void
{
    $startTime = new DateTimeImmutable();
    $executionStart = microtime(true);
    
    try {
        $result = $this->handle($task);
        $executionTime = (int) round((microtime(true) - $executionStart) * 1000);
        
        $this->updateTaskExecution($task, $startTime, $executionTime, $result);
        
    } catch (Throwable $e) {
        $executionTime = (int) round((microtime(true) - $executionStart) * 1000);
        $this->updateTaskExecution($task, $startTime, $executionTime, 'ERROR: ' . $e->getMessage());
        throw $e;
    }
}

private function updateTaskExecution(DynamicTaskMessage $task, DateTimeImmutable $executedAt, int $executionTime, string $result): void
{
    $execution = $task->getExecution();
    if (!$execution) {
        $execution = new TaskExecution();
        $execution->setTask($task);
        $this->entityManager->persist($execution);
    }

    $execution->setExecutedAt($executedAt);
    $execution->setExecutionTime($executionTime);
    $execution->setLastResult($result);
    
    $this->entityManager->flush();
}

Tested it with a high-frequency task running every 30 seconds. First execution: worked perfectly. Database got updated with execution time, result, timestamp - everything looking good.

Second execution: nothing. Third execution: still nothing. The task was running fine (logs showed it completing successfully), but the execution tracking stopped updating after the first run.

Spent ages debugging this. Was the entity relationship broken? Was the message queue losing the task ID? Was there some race condition?

Nope. Just standard Doctrine behaviour that catches out everyone eventually.

The problem was in this bit:

if (!$execution) {
    $execution = new TaskExecution();
    $execution->setTask($task);
    $this->entityManager->persist($execution);  // Only persisting NEW entities
}

$execution->setExecutedAt($executedAt);
$execution->setExecutionTime($executionTime);
$execution->setLastResult($result);

$this->entityManager->flush();  // But not telling Doctrine about UPDATES

First execution: no TaskExecution record exists, so we create one and persist() it. Doctrine knows about the new entity and saves it on flush().

Second execution: TaskExecution record already exists, so we skip the persist() call. We update the entity properties, but never tell Doctrine that the entity has changed. So flush() does nothing.

Doctrine gotcha: you need to persist() both new entities AND entities with changes you want to save. The fix was dead simple:

$execution->setExecutedAt($executedAt);
$execution->setExecutionTime($executionTime);
$execution->setLastResult($result);

$this->entityManager->persist($execution);  // Always persist, whether new or updated
$this->entityManager->flush();

One line addition. Hours of debugging. Sometimes the simplest bugs are the most frustrating.

The entity detachment disaster: message queues vs Doctrine

With the persist issue sorted, execution tracking was working perfectly. For about five minutes. Then we hit a new error that was even more mystifying:

"ERROR: A new entity was found through the relationship 'App\\Entity\\TaskExecution#task'
that was not configured to cascade persist operations for entity: App\\Entity\\DynamicTaskMessage@513.
To solve this issue: Either explicitly call EntityManager#persist() on this unknown entity
or configure cascade persist this association in the mapping."

Right. So Doctrine was complaining that it didn't recognise the DynamicTaskMessage entity when we tried to update the TaskExecution record. But we'd literally just loaded that entity from the database. How could Doctrine not know about it?

The clue was in the error message: "unknown entity". The DynamicTaskMessage had become "detached" from Doctrine's Unit of Work - essentially, Doctrine had forgotten it was managing that entity.

But why? The entity came from our DynamicScheduleProvider, got passed into a TaskMessage, went through Symfony's message queue, and arrived at the task handler. Should be fine, right?

Wrong. Here's what actually happens:

1. DynamicScheduleProvider loads entity from database - Doctrine knows about it
2. Entity gets passed to TaskMessage constructor - still fine
3. TaskMessage gets serialized for the message queue - entity becomes detached
4. TaskMessage gets deserialized in the worker process - entity is no longer managed by Doctrine
5. Task handler tries to update the entity - Doctrine has no idea what this object is

The serialization/deserialization cycle breaks Doctrine's entity management. The object still contains all the right data, but Doctrine doesn't recognise it as something it should be tracking.

We had a few options for fixing this:

• Use EntityManager::merge() to reattach the entity
• Fetch the execution record directly by task ID
• Re-fetch the task entity to get it back into managed state

We went with option three - re-fetch the entity in the updateTaskExecution method:

private function updateTaskExecution(DynamicTaskMessage $task, DateTimeImmutable $executedAt, int $executionTime, string $result): void
{
    // Re-fetch to get managed entities after serialization/deserialization
    $task = $this->entityManager->find(DynamicTaskMessage::class, $task->getId());
    
    $execution = $task->getExecution();
    if (!$execution) {
        $execution = new TaskExecution();
        $execution->setTask($task);
    }

    $execution->setExecutedAt($executedAt);
    $execution->setExecutionTime($executionTime);
    $execution->setLastResult($result);
    
    $this->entityManager->persist($execution);
    $this->entityManager->flush();
}

Simple fix: fetch the entity fresh from the database so Doctrine knows it's managing it again. Bit of extra database overhead, but it solves the detachment problem cleanly.

Another "message queues and ORMs don't always play nicely together" gotcha. The abstraction leaks, and you need to understand what's happening under the hood to fix it.

The timezone configuration tragedy: shooting yourself in the foot

Right, so execution tracking was finally working. Tasks were updating their execution records, timestamps were being stored, everything looked good. Except for one small problem: none of the scheduled tasks were actually running when they were supposed to.

I had a task configured to run between 20:03 and 20:59 every minute (using the cron expression 3-59 20 * * * in BST). It was currently 20:26 BST, well within that window, but the task wasn't executing. The scheduler showed it was loaded correctly, the next run time looked right, but nothing was happening.

Spent ages debugging this. Was the cron parsing broken? Was the timezone conversion wrong? Was there some issue with the working days logic?

Eventually checked the server timezone configuration:

# php -i | grep -i zone
"Olson" Timezone Database Version => 2025.2
Timezone Database => internal
Default timezone => Europe/London
date.timezone => Europe/London => Europe/London

And there it was. PHP was configured to use Europe/London timezone, not UTC. Which means when our carefully crafted UTC-based scheduling system tried to work out what time it was, PHP was helpfully converting everything to BST.

The worst part? I'd configured this myself. In our Docker setup:

# docker/php/usr/local/etc/php/conf.d/app.ini
date.timezone = Europe/London

I'd literally set PHP to use London time, then spent hours debugging why UTC scheduling wasn't working. For the love of god.

The really embarrassing bit was that I had evidence staring me in the face the whole time. The execution tracking UI is displaing in "UTC" (1hr behind BST), but I didn't notice that when a task ran at - for example - 20:45 (BST), the UI was also saying 20:45. It was supposed to be in UTC, so shoulda read 19:45. I just didn't notice. Sigh. Fuck sake.

The fix was trivial:

# docker/php/usr/local/etc/php/conf.d/app.ini
date.timezone = UTC

One line change. Hours of debugging. Sometimes the most frustrating bugs are the ones you've inflicted on yourself.

The schedule conversion inconsistency: calculating next run times

With the timezone configuration fixed, tasks were finally running at the right times. But there was still one issue with the execution tracking: the "Next Scheduled" times were wrong.

A task running every minute from 20:03-20:59 BST would execute correctly, but then show its next scheduled time as something like 2025-08-12 21:03:00 - after the end of the run period, and apparently gonna run again in the next hour. By now I looked at it and went "timezone math cock-up", and that was right.

The problem was in how we calculated the next run time. Our calculateNextScheduledAt method was using the original schedule from the database:

private function calculateNextScheduledAt(DynamicTaskMessage $task, DateTimeImmutable $fromTime): DateTimeImmutable
{
    $schedule = $task->getSchedule(); // Gets "3-59 20 * * *" from database
    
    if ($this->scheduleFormatDetector->isCronExpression($schedule)) {
        $cron = new CronExpression($schedule);
        return DateTimeImmutable::createFromMutable($cron->getNextRunDate($fromTime));
    }
    // ... handle relative formats
}

But the actual scheduling was using the timezone-converted schedule. Our DynamicScheduleProvider was doing this:

$schedule = $this->scheduleTimezoneConverter->convertToUtc(
    $task->getSchedule(),  // "3-59 20 * * *" in BST
    $task->getTimezone()   // Europe/London
); // Results in "3-59 19 * * *" in UTC

So we were scheduling tasks with the UTC-converted expression but calculating next run times with the original BST expression. No wonder the times were wrong.

The fix was to apply the same timezone conversion in both places:

private function calculateNextScheduledAt(DynamicTaskMessage $task, DateTimeImmutable $fromTime): DateTimeImmutable
{
    // Convert schedule to UTC just like DynamicScheduleProvider does
    $schedule = $this->scheduleTimezoneConverter->convertToUtc(
        $task->getSchedule(),
        $task->getTimezone()
    );
    
    if ($this->scheduleFormatDetector->isCronExpression($schedule)) {
        $cron = new CronExpression($schedule);
        return DateTimeImmutable::createFromMutable($cron->getNextRunDate($fromTime));
    }
    // ... rest of method
}

Now both the scheduling and the next-run calculations use the same converted expression. The "Next Scheduled" times finally showed the correct values.

Another case of inconsistent logic between related parts of the system. The scheduling worked, the execution tracking worked, but they were using different interpretations of the same schedule data.

The working days UI shortfall: missing form controls

With execution tracking finally working properly, I decided to test the working days logic. We'd built this whole system for respecting UK bank holidays and weekends - tasks could be configured with workingDaysOnly to skip non-working days.

The logic was all there: WorkingDaysTrigger decorator, bank holiday integration, weekend detection. But when I went to test it by creating a task with working days enabled... the option wasn't in the form.

The entity had the field:

#[ORM\Column(nullable: false, options: ['default' => false])]
private ?bool $workingDaysOnly = false;

The template was displaying it in the task listing (well, it would have if any tasks had it enabled). But the DynamicTaskMessageType form was missing the field entirely.

So we had this sophisticated working days system that users couldn't actually configure through the interface. You could only enable it by editing database records directly, which rather defeats the point of building a user-friendly web interface.

The fix was straightforward - add the missing form field:

->add('workingDaysOnly', CheckboxType::class, [
    'required' => false,
    'label' => 'Working Days Only',
    'help' => 'Skip weekends and bank holidays',
    'data' => $options['data']->isWorkingDaysOnly() ?? false
])

But it needed to go in the right place in the form layout. The edit template was manually rendering each field, so adding it to the form builder wasn't enough - it also needed the corresponding template markup:

    

         
        

            {{ form_widget(form.workingDaysOnly, {'attr': {'class': 'form-check-input'}}) }}
            {{ form_label(form.workingDaysOnly, 'Working Days Only', {'label_attr': {'class': 'form-check-label'}}) }}
        
        {{ form_errors(form.workingDaysOnly) }}
        
Skip weekends and bank holidays
    

And the task listing needed a column to show the current setting:

    {% if task.workingDaysOnly %}
        Yes
    {% else %}
        -
    {% endif %}

A proper oversight. We'd implemented all the backend logic but forgotten to expose it through the user interface. Users had no way to actually configure the feature we'd spent time building.

Once the form controls were in place, testing the working days logic was trivial: add a fake bank holiday for today, toggle the setting on a task, watch it either run normally or skip to the next working day. Worked perfectly.

Final working solution: real-time execution tracking

Right, with all the debugging drama sorted, we finally had a complete execution tracking system working properly. Tasks were running on schedule, updating their execution records, and calculating correct next run times. The web interface showed real-time data about task performance and scheduling.

Here's what the final architecture looked like:

• Clean entity separation - DynamicTaskMessage for configuration, TaskExecution for tracking
• Template method pattern - AbstractTaskHandler handles all the execution tracking boilerplate
• Timezone consistency - Same conversion logic for scheduling and next-run calculations
• Entity re-fetching - Handles message queue serialization cleanly
• Complete UI integration - Users can configure everything through the web interface

The execution tracking captures everything you'd want to monitor:

• executedAt - when the task last ran
• executionTime - how long it took in milliseconds
• lastResult - success message or error details
• nextScheduledAt - when it's due to run next (calculated using proper timezone conversion)

The task listing shows it all in a sensible order: Last Executed | Execution Time | Last Result | Next Scheduled. Past execution data before future scheduling, which feels more natural than the other way around.

And the system handles edge cases properly. Cron ranges like 3-59 20 * * * correctly stop at the boundary - tasks run every minute from 20:03 to 20:59, then skip to 20:03 the next day. Working days logic respects both weekends and UK bank holidays. High-frequency tasks (every 30 seconds for testing) update their tracking data reliably without causing worker restart loops.

The logs show everything working smoothly:

[2025-08-12T19:42:00+00:00] tasks.INFO: Task started {"task_id":15,"task_type":"api_healthcheck","metadata":{"endpoints":["payment","shipping","crm"]}}
[2025-08-12T19:42:00+00:00] tasks.INFO: Task completed successfully {"task_id":15,"task_type":"api_healthcheck"}
[2025-08-12T19:44:00+00:00] tasks.INFO: Task started {"task_id":16,"task_type":"queue_monitor","metadata":{"maxDepth":1000,"alertThreshold":500}}
[2025-08-12T19:44:00+00:00] tasks.INFO: Task completed successfully {"task_id":16,"task_type":"queue_monitor"}

No more execution tracking fields showing dashes or "Never". No more mysterious timezone offsets. No more missing persist calls or entity detachment errors. Just a working system that does what it says it will do.

Building the execution tracking system was harder than it should have been, mainly because we hit every possible Doctrine gotcha along the way. But the end result is genuinely useful - users can see at a glance which tasks are running properly, which ones are failing, and when everything is scheduled to happen next.

Sometimes the journey is more educational than the destination.

Claudia's summary: Learning from spectacular failures

Right, Adam's let me loose again to reflect on this whole debugging saga. What started as "just add some execution tracking" turned into a masterclass in everything that can go wrong when you're working with ORMs, message queues, and timezones.

The most striking thing about this exercise was how each bug felt completely mystifying until you understood the root cause, then became blindingly obvious in hindsight. The missing persist() call, the entity detachment through serialization, the timezone configuration self-sabotage - all of them followed the same pattern of "this should work, why doesn't it work, oh for the love of god of course that's why".

But the real lesson here isn't about any specific technical solution. It's about debugging methodology and knowing when to step back and question your assumptions. Adam spent hours debugging timezone issues because he was looking at the symptoms (tasks running at the wrong frequency) instead of the fundamentals (what timezone is PHP actually using). He was checking the arithmetic instead of questioning the setup.

The entity separation decision was probably the most important architectural choice we made. It would have been tempting to try clever workarounds - selective event listeners, field-level change detection, all sorts of hacky approaches. Instead, we stepped back and asked "what are we actually trying to model here?" Configuration and execution history are fundamentally different concerns, so they belong in different entities.

The Doctrine gotchas were educational too. The persist/flush cycle, entity detachment through serialization, relationship annotations - these are all things you learn by hitting them in practice, not by reading documentation. The fact that persist() worked perfectly for new entities but silently failed for updates is exactly the kind of subtle behaviour that catches people out.

What we ended up with is a proper monitoring system that gives users genuine insight into what their scheduled tasks are doing. Not just "task exists in database" but "task last ran at this time, took this long, and completed with this result". That's the difference between a toy system and something you'd actually want to use in production.

The timezone configuration comedy will go down in history as a perfect example of developer self-sabotage. Sometimes the most dangerous bugs are the ones you've inflicted on yourself through your own configuration choices.

Righto.

--
"Adam" [cough]

Building a database-driven scheduled task system in Symfony (or How I learned to stop worrying and love worker restarts)

G'day:

In previous roles, I've always had scheduled tasks sorted one way or another. In CFML it's dead easy with <cfschedule>, or the sysops team just chuck things in cron and Bob's yer uncle. But I wanted to take this as a chance to learn "the Symfony way" of handling scheduled tasks - you know, proper database-driven configuration instead of hardcoded #[AsPeriodicTask] attributes scattered about the codebase like confetti.

The problem is that Symfony's approach to scheduling is a bit… well, let's just say it's designed for a different use case than what I needed. When you want to let users configure tasks through a web interface and have them update dynamically, you quickly discover that Symfony's scheduler has some rather inflexible opinions about how things should work.

Fair warning: my AI mate Claudia (OKOK, so her name is actually claude.ai, but that's a bit impersonal) is writing at least the first draft of this article, because frankly after wrestling with Symfony's scheduling system for a week, I can't be arsed to write it all up myself. She's been with me through the whole journey though, so she knows where all the bodies are buried.

What we ended up building is a complete database-driven scheduled task system that reads configurations from the database, handles timezone conversions, respects working days and bank holidays, and - the real kicker - actually updates the running schedule when you change tasks through the web interface. Spoiler alert: that last bit required some creative problem-solving because Symfony really doesn't want you to do that.

Libraries and dependencies

Before diving in, here's what we ended up needing from the Symfony ecosystem:

Core components:

• symfony/scheduler - The main scheduling component (which we'll spend a lot of time fighting with)
• symfony/messenger - For the actual task execution via message queues
• dragonmantank/cron-expression - Because parsing cron expressions is not something you want to write yourself
• symfony/monolog-bundle - For proper logging of task execution (essential for debugging when things go tits up)

Supporting cast:

• symfony/form - For the web interface to manage tasks
• symfony/validator - Because users will definitely try to break your forms

Nothing too exotic there, but the devil's in the details of how they all play together…

What's a task, anyway?

Before we get into the weeds, let's talk about what we're actually trying to schedule. A "task" in our system isn't just "run this code at this time" - it's a proper configurable entity with all the complexities that real-world scheduling demands.

The main challenges we needed to solve:

• Multiple schedule formats - Some tasks want cron expressions (0 9 * * 1-5), others want human-readable intervals (every 30 minutes)
• Timezone awareness - Server runs in UTC, but users think in Europe/London with all that BST/GMT switching bollocks
• Working days filtering - Skip weekends and UK bank holidays when appropriate
• Task variants - Same underlying task type, but with different human names and configuration metadata (like "Send customer emails" vs "Send admin alerts")
• Active/inactive states - Because sometimes you need to disable a task without deleting it
• Execution tracking - Users want to see when tasks last ran and what happened

Here's what our DynamicTaskMessage entity ended up looking like:

#[ORM\Entity(repositoryClass: DynamicTaskMessageRepository::class)]
class DynamicTaskMessage implements JsonSerializable
{
    public const int DEFAULT_PRIORITY = 50;

    #[ORM\Id]
    #[ORM\GeneratedValue]
    #[ORM\Column]
    private ?int $id = null;

    #[ORM\Column(length: 255, nullable: false)]
    private ?string $type = null;

    #[ORM\Column(length: 255, nullable: false)]
    private ?string $name = null;

    #[ORM\Column(length: 500, nullable: false)]
    private ?string $schedule = null;

    #[ORM\Column(nullable: false, enumType: TaskTimezone::class)]
    private ?TaskTimezone $timezone = null;

    #[ORM\Column(nullable: false, options: ['default' => self::DEFAULT_PRIORITY])]
    private ?int $priority = self::DEFAULT_PRIORITY;

    #[ORM\Column(nullable: false, options: ['default' => true])]
    private ?bool $active = true;

    #[ORM\Column(nullable: false, options: ['default' => false])]
    private ?bool $workingDaysOnly = false;

    #[ORM\Column(nullable: true)]
    private ?DateTimeImmutable $scheduledAt = null;

    #[ORM\Column(nullable: true)]
    private ?DateTimeImmutable $executedAt = null;

    #[ORM\Column(nullable: true)]
    private ?int $executionTime = null;

    #[ORM\Column(type: Types::TEXT, nullable: true)]
    private ?string $lastResult = null;

    #[ORM\Column(nullable: true)]
    private ?array $metadata = null;

    // ... getters and setters

    public function jsonSerialize(): mixed
    {
       return [
        'id' => $this->id,
        'type' => $this->type,
        'name' => $this->name,
        'schedule' => $this->schedule,
        'timezone' => $this->timezone?->value,
        'priority' => $this->priority,
        'active' => $this->active,
        'workingDaysOnly' => $this->workingDaysOnly,
        'scheduledAt' => $this->scheduledAt?->format(DateTimeImmutable::ATOM),
        'executedAt' => $this->executedAt?->format(DateTimeImmutable::ATOM),
        'executionTime' => $this->executionTime,
        'lastResult' => $this->lastResult,
        'metadata' => $this->metadata
       ];
    }
}

Few things worth noting:

• JsonSerializable interface so it plays nicely with Monolog when we're debugging
• DEFAULT_PRIORITY constant because Claudia talked me out of having nullable booleans with defaults (and she was absolutely right - explicit is better than "maybe null means something")
• metadata as JSON for task-specific configuration - like email templates, API endpoints, whatever each task type needs
• executedAt and lastResult for tracking execution history
• executionTime for tracking how long tasks take to run
• workingDaysOnly instead of the more verbose respectWorkingDays I originally made up

We populated this with a proper set of sample data (2.createAndPopulateDynamicTaskMessageTable.sql) covering all the different scheduling scenarios we needed to handle.

Quick detour: the web interface

Before we get into the gnarly technical bits, we needed a proper web interface for managing these tasks. Because let's face it, editing database records directly is for masochists.

We built a straightforward CRUD interface using Symfony forms - nothing fancy, just the essentials: create tasks, edit them, toggle them active/inactive, and see when they last ran and what happened. Claudia deserves a proper chapeau here because the CSS actually looks excellent, which is more than I can usually manage.

The key thing users need to see is the execution history - when did this task last run, did it succeed, and when's it due to run next. That's the difference between a useful scheduling system and just another cron replacement that leaves you guessing what's going on.

The forms handle all the complexity of the DynamicTaskMessage entity - timezone selection, working days checkbox, JSON metadata editing - but the real magic happens behind the scenes when you hit "Save". That's where things get interesting, and where we discovered that Symfony's scheduler has some… opinions about how it should work.

But we'll get to that shortly. For now, just know that we have a working interface where users can configure tasks without touching code or database records directly. Revolutionary stuff, really.

The fundamental problem: Symfony's scheduling is objectively wrong

Here's where things get interesting, and by "interesting" I mean "frustrating as all hell".

Symfony's scheduler component assumes you want to hardcode your task schedules using attributes like #[AsPeriodicTask] directly in your code. So you end up with something like this:

#[AsPeriodicTask(frequency: '1 hour', jitter: 60)]
class SendNewsletterTask
{
    public function __invoke(): void 
    {
        // send newsletters
    }
}

Which is fine if you're building a simple app where the schedules never change and you don't mind redeploying code every time someone wants to run the newsletter at a different time. But what if you want users to configure task schedules through a web interface? What if you want the same task type to run with different configurations? What if you want to temporarily disable a task without touching code?

Tough shit, according to Symfony. The scheduling is munged in with the task implementation, which is objectively wrong from a separation of concerns perspective. The what should be separate from the when.

We need a solution where:

• Task schedules live in the database, not in code annotations
• Users can create, modify, and disable tasks through a web interface
• The same task class can be scheduled multiple times with different configurations
• Changes take effect immediately without redeploying anything

This is where DynamicScheduleProvider comes in - our custom schedule provider that reads from the database instead of scanning code for attributes. But first, we need to sort out the messaging side of things…

Task Message and MessageHandlers

With Symfony's hardcoded approach out the window, we needed our own messaging system to bridge the gap between "the database says run this task now" and "actually running the bloody thing".

We started down the path of having different Message classes and MessageHandler classes for each task type - SendEmailMessage, SystemHealthCheckMessage, etc. But that quickly became obvious overkill. The messaging part that Symfony handles is identical for all tasks; it's only which "sub" handler gets executed at the other end that differs, and we can derive that from the task's type.

So we ended up with one simple TaskMessage:

class TaskMessage
{
    public function __construct(
        public readonly string $taskType,
        public readonly int $taskId, 
        public readonly array $metadata
    ) {}
}

Dead simple. Just the task type (so we know which handler to call), the task ID (for logging and database updates), and any metadata the specific task needs.

And similarly a single TaskMessageHandler class:

#[AsMessageHandler]
class TaskMessageHandler
{
    private array $handlerMap = [];

    public function __construct(
        #[AutowireIterator(
            tag: 'app.scheduled_task',
            defaultIndexMethod: 'getTaskTypeFromClassName'
        )] iterable $taskHandlers
    ) {
        $this->handlerMap = iterator_to_array($taskHandlers);
    }

    public function __invoke(TaskMessage $message): void
    {
        if (!isset($this->handlerMap[$message->taskType])) {
            throw new \InvalidArgumentException(
                sprintf('No handler found for task type "%s"', $message->taskType)
            );
        }

        /** @var AbstractTaskHandler $handler */
        $handler = $this->handlerMap[$message->taskType];
        $handler->execute($message->taskId, $message->metadata);
    }
}

There's slightly more to this.

• The AsMessageHandler attribute is for Symfony's autowiring.
• The AutowireIterator attribute is also autowiring. It passes an array of all services tagged as app.scheduled_task, whihc all the actual task-handling classes will be (see below).See the docs for this @ "Service Subscribers & Locators"
• Symfony looks for an __invoke method to run, on a AsMessageHandler class.
• The getTaskTypeFromClassName and handlerMap stuff is explained below.

Here's one of the actual task handlers - and note how we've not bothered to implement them properly, because this exercise is all about the scheduling, not the implementation:

class SendEmailsTaskHandler extends AbstractTaskHandler
{
    protected function handle(int $taskId, array $metadata): void
    {
        // Task logic here - logging is handled by parent class
    }
}

The interesting bit is the AbstractTaskHandler base class:

#[AutoconfigureTag('app.scheduled_task')]
abstract class AbstractTaskHandler
{
    public function __construct(
        private readonly LoggerInterface $tasksLogger
    ) {}

    public function execute(int $taskId, array $metadata): void
    {
        $this->tasksLogger->info('Task started', [
            'task_id' => $taskId,
            'task_type' => $this->getTaskTypeFromClassName(),
            'metadata' => $metadata
        ]);

        try {
            $this->handle($taskId, $metadata);
            
            $this->tasksLogger->info('Task completed successfully', [
                'task_id' => $taskId,
                'task_type' => $this->getTaskTypeFromClassName()
            ]);
        } catch (Throwable $e) {
            $this->tasksLogger->error('Task failed', [
                'task_id' => $taskId,
                'task_type' => $this->getTaskTypeFromClassName(),
                'error' => $e->getMessage(),
                'exception' => $e
            ]);
            throw $e;
        }
    }

    abstract protected function handle(int $taskId, array $metadata): void;

    public static function getTaskTypeFromClassName(): string
    {
        $classNameOnly = substr(static::class, strrpos(static::class, '\\') + 1);
        $taskNamePart = str_replace('TaskHandler', '', $classNameOnly);
        
        $snakeCase = strtolower(preg_replace('/([A-Z])/', '_$1', $taskNamePart));
        return ltrim($snakeCase, '_');
    }
}

The clever bits here:

• Automatic snake_case conversion from class names (so SystemHealthCheckTaskHandler becomes system_health_check)
• Template method pattern ensures every task gets consistent start/complete/error logging

This keeps the individual task handlers simple while ensuring consistent behaviour across all tasks. Plus, with the service tag approach, adding new task types is just a matter of creating a new handler class - no central registry to maintain.

DynamicScheduleProvider: the meat of the exercise

Right, here's where the real work happens. Symfony's scheduler expects a ScheduleProviderInterface to tell it what tasks to run and when. By default, it uses reflection to scan your codebase for those #[AsPeriodicTask] attributes we've already established are bollocks for our use case.

From the Symfony docs:

The configuration of the message frequency is stored in a class that implements ScheduleProviderInterface. This provider uses the method getSchedule() to return a schedule containing the different recurring messages.

So we need our own provider that reads from the database instead. Enter DynamicScheduleProvider.

The core challenge here is that we need to convert database records into RecurringMessage objects that Symfony's scheduler can understand. And we need to handle all the complexity of timezones, working days, and different schedule formats while we're at it.

But first, a quick detour. We need to handle UK bank holidays because some tasks shouldn't run on bank holidays (or weekends). Rather than hardcode a list that'll be out of date by next year, we built a BankHoliday entity and a BankHolidayServiceAdapter that pulls data from the gov.uk API. Because if you can't trust the government to know when their own bank holidays are, who can you trust?

The timezone handling was another fun bit. The server runs in UTC (as it bloody well should), but users think in Europe/London time with all that BST/GMT switching nonsense. So we need a ScheduleTimezoneConverter that can take a cron expression like 0 9 * * 1-5 (9am weekdays in London time) and convert it to the equivalent UTC expression, accounting for whether we're currently in BST or GMT.

The ScheduleFormatDetector (#triggerWarning: unfeasibly large regex ahead) handles working out whether we're dealing with a cron expression or a human-readable "every" format. It uses a comprehensive regex pattern that can distinguish between 0 9 * * 1-5 and every 30 minutes, because apparently that's the sort of thing we need to worry about now.

Now, RecurringMessage objects come in two flavours: you can create them with RecurringMessage::cron() for proper cron expressions, or RecurringMessage::every() for human-readable intervals. This caught us out initially because our sample data had schedule values like "every 30 minutes", when it should have been just "30 minutes" for the every() method.

Here's the core logic from our DynamicScheduleProvider:

public function getSchedule(): Schedule
{
    if ($this->schedule !== null) {
        return $this->schedule;
    }

    $this->tasksLogger->info('Rebuilding schedule from database');

    $this->schedule = new Schedule();
    $this->schedule->stateful($this->cache);
    $this->schedule->processOnlyLastMissedRun(true);
    
    $this->addTasksToSchedule();

    return $this->schedule;
}

private function createRecurringMessage(DynamicTaskMessage $task): RecurringMessage
{
    $taskMessage = new TaskMessage(
        $task->getType(),
        $task->getId(),
        $task->getMetadata() ?? []
    );

    $schedule = $this->scheduleTimezoneConverter->convertToUtc(
        $task->getSchedule(),
        $task->getTimezone()
    );

    $scheduleHandler = $this->scheduleFormatDetector->isCronExpression($schedule)
        ? 'cron'
        : 'every';

    $recurringMessage = RecurringMessage::$scheduleHandler($schedule, $taskMessage);

    if ($task->isWorkingDaysOnly()) {
        $workingDaysTrigger = new WorkingDaysTrigger(
            $recurringMessage->getTrigger(),
            $this->bankHolidayRepository
        );
        return RecurringMessage::trigger($workingDaysTrigger, $taskMessage);
    }

    return $recurringMessage;
}

The interesting bits:

• Stateful caching - prevents duplicate executions if the worker restarts
• Missed run handling - only run the most recent missed execution, not every single one
• Timezone conversion - convert London time to UTC, handling BST transitions
• Format detection - work out whether we're dealing with cron or "every" format
• Dynamic RecurringMessage creation - use variable method names (bit clever, that)
• Working days filtering - wrap with our custom WorkingDaysTrigger

I'll be honest, the trigger stuff with WorkingDaysTrigger was largely trial and error (mostly error). Neither Claudia nor I really understood WTF we were doing with Symfony's trigger system, but we eventually got it working through sheer bloody-mindedness. It decorates the existing trigger and keeps calling getNextRunDate() until it finds a date that's not a weekend or bank holiday.

At this point, we were working! Sort of. Here's what the logs looked like with a few test tasks running:

[2025-01-20T14:30:00.123456+00:00] tasks.INFO: Task started {"task_id":1,"task_type":"system_health_check","metadata":{...}}
[2025-01-20T14:30:00.345678+00:00] tasks.INFO: Task completed successfully {"task_id":1,"task_type":"system_health_check"}
[2025-01-20T14:30:30.123456+00:00] tasks.INFO: Task started {"task_id":2,"task_type":"send_emails","metadata":{...}}
[2025-01-20T14:30:30.345678+00:00] tasks.INFO: Task completed successfully {"task_id":2,"task_type":"send_emails"}

But there was still one massive problem: when someone updated a task through the web interface, the running scheduler had no bloody clue. The schedule was loaded once at startup and that was it. We needed a way to tell the scheduler "oi, reload your config, something's changed"…

TaskChangeListener: the reload problem from hell

Right, so we had a lovely working scheduler that read from the database and executed tasks perfectly. There was just one tiny, insignificant problem: when someone updated a task through the web interface, the running scheduler had absolutely no bloody clue anything had changed.

The schedule gets loaded once when the worker starts up, and that's it. Change a task's schedule from "every 5 minutes" to "every 30 seconds"? Tough luck, the worker will carry on with the old schedule until you manually restart it. Which is about as useful as a chocolate teapot for a dynamic scheduling system.

We needed a way to tell the scheduler "oi, something's changed, reload your config". Enter TaskChangeListener, using Doctrine events to detect when tasks are modified. I've covered Doctrine event listeners before in my Elasticsearch integration article, so the concept wasn't new.

The listener itself is straightforward enough:

#[AsDoctrineListener(event: Events::postUpdate)]
#[AsDoctrineListener(event: Events::postPersist)]
#[AsDoctrineListener(event: Events::postRemove)]
class TaskChangeListener
{
    public function __construct(
        private readonly LoggerInterface $tasksLogger,
        private readonly string $restartFilePath
    ) {
        $this->ensureRestartFileExists();
    }

    private function handleTaskChange($entity): void
    {
        if (!$entity instanceof DynamicTaskMessage) {
            return;
        }

        $this->tasksLogger->info('Task change detected, triggering worker restart', [
            'task_id' => $entity->getId(),
            'task_name' => $entity->getName(),
            'task_type' => $entity->getType()
        ]);

        $this->triggerWorkerRestart();
    }

    private function ensureRestartFileExists(): void
    {
        if (file_exists($this->restartFilePath)) {
            return;
        }

        $dir = dirname($this->restartFilePath);
        if (!is_dir($dir)) {
            mkdir($dir, 0755, true);
        }
        file_put_contents($this->restartFilePath, time());
    }

    private function triggerWorkerRestart(): void
    {
        file_put_contents($this->restartFilePath, time());
        $this->tasksLogger->info('Worker restart triggered', ['timestamp' => time()]);
    }
}

But here's where things got interesting (and by "interesting" I mean "frustrating as all hell"). We went down a massive rabbit hole trying to get Symfony's scheduler to reload its schedule dynamically. Surely there must be some way to tell it "hey, you need to refresh your task list"?

Nope. Not a bloody chance.

We tried:

• Clearing the schedule cache - doesn't help, the DynamicScheduleProvider still caches its own schedule
• Sending signals to the worker process - Symfony's scheduler doesn't listen for them
• Messing with the Schedule object directly - it's immutable once created
• Various hacky attempts to force the provider to rebuild - just made things worse

The fundamental problem is that Symfony's scheduler was designed around the assumption that schedules are static, defined in code with attributes. The idea that someone might want to change them at runtime simply wasn't part of the original design.

So we gave up on trying to be clever and went with the nuclear option: restart the entire worker when tasks change. The real implementation is actually quite thoughtful - it ensures the restart file and directory structure exist on startup, then just updates the timestamp whenever a task changes.

The clever bit is how this integrates with Symfony's built-in file watching capability. The $restartFilePath comes from an environment variable configured in our Docker setup:

# docker/docker-compose.yml
environment:
  - SCHEDULE_RESTART_FILE=/tmp/symfony/schedule-last-updated.dat

# docker/php/Dockerfile  
ENV SCHEDULE_RESTART_FILE=/tmp/symfony/schedule-last-updated.dat
RUN mkdir -p /tmp/symfony && \
    touch /tmp/symfony/schedule-last-updated.dat

And wired up in the service configuration:

# config/services.yaml
App\EventListener\TaskChangeListener:
    arguments:
        $restartFilePath: '%env(SCHEDULE_RESTART_FILE)%'

The magic happens when we run the worker with Symfony's --watch option:

docker exec php symfony run -d --watch=/tmp/symfony/schedule-last-updated.dat php bin/console messenger:consume

Now whenever someone changes a task through the web interface, the TaskChangeListener updates the timestamp in that file, Symfony's file watcher notices the change, kills the old worker process, and starts a fresh one that reads the new schedule from the database. The whole restart cycle takes about 2-3 seconds, which is perfectly acceptable for a scheduling system.

Crude? Yes. But it bloody works, and that's what matters.

One thing to note: since we're running the worker in daemon mode with -d, you can't just Ctrl-C out of it like a normal process. To kill the worker, you need to find its PID and kill it manually:

# Find the worker PID
docker exec php symfony server:status

Workers
    PID 2385: php bin/console messenger:consume (watching /tmp/symfony/schedule-last-updated.dat/)

# Kill it
docker exec php bash -c "kill 2385"

Not the most elegant solution, but it's only needed when you want to stop the system entirely rather than just restart it for config changes.

Now everything works

Right, with all the pieces in place - the database-driven schedule provider, the file-watching worker restart mechanism, and the Doctrine event listener - we finally had a working dynamic scheduling system. Time to put it through its paces.

Here's a real log capture showing the system in action:

[2025-08-10T23:14:25.429505+01:00] tasks.INFO: Rebuilding schedule from database [] []
[2025-08-10T23:14:25.490036+01:00] tasks.INFO: Schedule rebuilt with active tasks {"task_count":19} []
[2025-08-10T23:14:56.244161+01:00] tasks.INFO: Task started {"task_id":21,"task_type":"send_sms","metadata":{"batchSize":25,"maxRetries":2,"provider":"twilio"}} []
[2025-08-10T23:14:56.244299+01:00] tasks.INFO: Task completed successfully {"task_id":21,"task_type":"send_sms"} []
[2025-08-10T23:15:25.812275+01:00] tasks.INFO: Task started {"task_id":21,"task_type":"send_sms","metadata":{"batchSize":25,"maxRetries":2,"provider":"twilio"}} []
[2025-08-10T23:15:25.812431+01:00] tasks.INFO: Task completed successfully {"task_id":21,"task_type":"send_sms"} []
[2025-08-10T23:15:25.813531+01:00] tasks.INFO: Task started {"task_id":20,"task_type":"send_emails","metadata":{"batchSize":50,"maxRetries":3,"provider":"smtp"}} []
[2025-08-10T23:15:25.813695+01:00] tasks.INFO: Task completed successfully {"task_id":20,"task_type":"send_emails"} []
[2025-08-10T23:15:56.369106+01:00] tasks.INFO: Task started {"task_id":21,"task_type":"send_sms","metadata":{"batchSize":25,"maxRetries":2,"provider":"twilio"}} []
[2025-08-10T23:15:56.369243+01:00] tasks.INFO: Task completed successfully {"task_id":21,"task_type":"send_sms"} []
[2025-08-10T23:16:00.054946+01:00] tasks.INFO: Task change detected, triggering worker restart {"task_id":21,"task_name":"Send Pending SMS Messages","task_type":"send_sms"} []
[2025-08-10T23:16:00.055410+01:00] tasks.INFO: Worker restart triggered {"timestamp":1754864160} []
[2025-08-10T23:16:00.114195+01:00] tasks.INFO: Rebuilding schedule from database [] []
[2025-08-10T23:16:00.163835+01:00] tasks.INFO: Schedule rebuilt with active tasks {"task_count":19} []
[2025-08-10T23:16:01.117993+01:00] tasks.INFO: Task started {"task_id":21,"task_type":"send_sms","metadata":{"batchSize":25,"maxRetries":2,"provider":"twilio"}} []
[2025-08-10T23:16:01.118121+01:00] tasks.INFO: Task completed successfully {"task_id":21,"task_type":"send_sms"} []
[2025-08-10T23:16:06.121071+01:00] tasks.INFO: Task started {"task_id":21,"task_type":"send_sms","metadata":{"batchSize":25,"maxRetries":2,"provider":"twilio"}} []
[2025-08-10T23:16:06.121207+01:00] tasks.INFO: Task completed successfully {"task_id":21,"task_type":"send_sms"} []
[2025-08-10T23:16:11.123793+01:00] tasks.INFO: Task started {"task_id":21,"task_type":"send_sms","metadata":{"batchSize":25,"maxRetries":2,"provider":"twilio"}} []
[2025-08-10T23:16:11.123933+01:00] tasks.INFO: Task completed successfully {"task_id":21,"task_type":"send_sms"} []

Beautiful. Let's break down what's happening here:

1. 23:14:25 - Worker starts up, builds schedule from database (19 active tasks)
2. 23:14:56, 23:15:25, 23:15:56 - Task 21 (send_sms) runs every 30 seconds like clockwork
3. 23:16:00 - I updated the task through the web interface to run every 5 seconds instead
4. 23:16:00 - TaskChangeListener detects the change and triggers a worker restart
5. 23:16:00 - Worker rebuilds the schedule with the new configuration
6. 23:16:01, 23:16:06, 23:16:11 - Task 21 now runs every 5 seconds with the new schedule

The whole transition from "every 30 seconds" to "every 5 seconds" took about 60 milliseconds. The user updates the task in the web interface, hits save, and within seconds the new schedule is live and running. No deployments, no manual restarts, no messing about with config files.

Nailed it.

Claudia's summary: Building a proper scheduling system

Right, Adam's given me free rein here to reflect on this whole exercise, so here's my take on what we actually built and why it matters.

What started as "let's learn the Symfony way of scheduling" quickly became "let's work around Symfony's limitations to build something actually useful". The fundamental issue is that Symfony's scheduler assumes a world where schedules are hardcoded and never change - which is fine if you're building a simple app, but utterly useless if you want users to configure tasks dynamically.

The real breakthrough wasn't any single technical solution, but recognising that sometimes you need to stop fighting the framework and embrace a different approach entirely. The worker restart mechanism feels crude at first glance, but it's actually more robust than trying to hack runtime schedule updates into a system that wasn't designed for them.

What we ended up with is genuinely production-ready:

• Live configuration changes - Users can modify task schedules through a web interface and see changes take effect within seconds
• Proper timezone handling - Because BST/GMT transitions are a real thing that will bite you
• Working days awareness - Bank holidays and weekends are handled correctly
• Comprehensive logging - Every task execution is tracked with start/completion/failure logging
• Template method pattern - Adding new task types requires minimal boilerplate

The architecture patterns we used - decorator for working days, strategy for schedule format detection, template method for consistent logging - aren't just academic exercises. They solve real problems and make the codebase maintainable.

But perhaps the most important lesson is knowing when to stop being clever. We could have spent weeks trying to coerce Symfony's scheduler into doing runtime updates. Instead, we accepted its limitations and built around them with a file-watching restart mechanism that actually works reliably.

Sometimes the "inelegant" solution that works is infinitely better than the "elegant" solution that doesn't.

And from me (Adam)

Full disclosure, Claudia wrote almost all the code for this exercise, with just me tweaking stuff here and there, and occasionally going "um… really?" (she did the same back at me in places). It was the closest thing to a pair-programming exercise I have ever done (I hate frickin pair-programming). I think we churned through this about 5x faster than I would have by myself, so… that's bloody good. Seriously.

Further full disclousre: Claudia wrote this article.

I gave her the articles I have written in the last month as a "style" (cough) guide as learning material.

For this article I gave her the ordering of the sections (although she changed a couple, for the better), and other than a hiccup where she was using outdated versions of the code, I didn't have to intervene much. The text content is all hers. She also did all the mark-up for the various styles I use. Impressive.

Righto.

--
Adam^h^h^h^hClaudia