Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve cron error logging #32690

Open
wants to merge 42 commits into
base: 2.4-develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
d39c093
Produce success message only for humans
fredden Apr 6, 2021
cd7c499
Catch STDERR as well as STDOUT
fredden Apr 6, 2021
0fe0e18
Log output of background processes
fredden Apr 6, 2021
0e2c0b8
Update test for new functionality
fredden Apr 7, 2021
6e85c80
Correct comment
fredden Apr 7, 2021
42d783c
Add ext-posix to root composer.json
fredden Apr 8, 2021
5470014
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden Apr 8, 2021
60d4065
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden Aug 6, 2021
028e098
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden Oct 21, 2022
1f902da
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden Nov 4, 2022
c59aa26
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden Jan 25, 2023
184e9e9
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden Mar 3, 2023
1a6ae2b
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden Mar 14, 2023
5af0492
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden Apr 4, 2023
5e7af7c
Merge branch '2.4-develop' into cron-log
engcom-Hotel Apr 21, 2023
e63de37
Merge branch '2.4-develop' into cron-log
engcom-Hotel Apr 25, 2023
365236a
Fix unit test failure
fredden Apr 26, 2023
a1d3859
Ignore intentional use of discouraged function
fredden May 2, 2023
4ff438d
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden May 16, 2023
13bbeb1
Replace posix_isatty() with stream_isatty()
fredden May 16, 2023
17638f5
Move Framework modifications into module
fredden May 16, 2023
485869e
Adhere to coding standard
fredden May 16, 2023
64df5bb
Correct namespace
fredden May 16, 2023
8029ce9
Add 'declare(strict_types=1);'
fredden May 16, 2023
752821b
Move global/api virtualType to app/etc/di.xml
fredden May 17, 2023
6a5917e
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden May 17, 2023
bc298aa
Merge remote-tracking branch 'magento/2.4-develop' into cron-log
engcom-Charlie May 22, 2023
e44ac48
Draft:Increase the page load time
engcom-Charlie May 24, 2023
3925deb
Revert "Draft:Increase the page load time"
fredden May 24, 2023
4aa88cf
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden May 24, 2023
cdc2a75
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden Jul 17, 2023
986eb5b
Merge remote-tracking branch 'upstream/2.4-develop' into cron-log
fredden Dec 9, 2023
2506852
Merge branch '2.4-develop' into cron-log
fredden Feb 29, 2024
a9f9be2
Merge branch '2.4-develop' into cron-log
engcom-Charlie Mar 6, 2024
3a7761f
Merge branch '2.4-develop' into cron-log
engcom-Charlie Mar 7, 2024
15075d7
Merge branch '2.4-develop' into cron-log
engcom-Charlie Mar 8, 2024
4e3a399
Merge branch '2.4-develop' into cron-log
engcom-Charlie Apr 4, 2024
c0c7bed
Merged latest codebase
Jun 20, 2024
14868e8
Merge branch '2.4-develop' into cron-log
engcom-Charlie Oct 9, 2024
e633b7f
Merge branch '2.4-develop' into cron-log
engcom-Charlie Nov 21, 2024
9886e23
Fixed Static test failures
Nov 22, 2024
37d1481
Fixed Static test
engcom-Charlie Nov 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion app/code/Magento/Cron/Console/Command/CronCommand.php
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
<?php

/**
* Copyright © Magento, Inc. All rights reserved.
* See COPYING.txt for license details.
Expand Down Expand Up @@ -135,10 +136,15 @@ protected function execute(InputInterface $input, OutputInterface $output)
$params[ProcessCronQueueObserver::STANDALONE_PROCESS_STARTED] = $bootstrapOptionValue;
}
}

/** @var Cron $cronObserver */
$cronObserver = $objectManager->create(Cron::class, ['parameters' => $params]);
$cronObserver->launch();
$output->writeln('<info>' . 'Ran jobs by schedule.' . '</info>');

// phpcs:ignore Magento2.Functions.DiscouragedFunction.Discouraged,Magento2.Exceptions.TryProcessSystemResources.MissingTryCatch
if (stream_isatty(STDOUT)) {
$output->writeln('<info>' . 'Ran jobs by schedule.' . '</info>');
}

return Cli::RETURN_SUCCESS;
}
Expand Down
8 changes: 5 additions & 3 deletions app/code/Magento/Cron/Observer/ProcessCronQueueObserver.php
Original file line number Diff line number Diff line change
Expand Up @@ -278,10 +278,12 @@ function ($a, $b) {
&& $this->getCronGroupConfigurationValue($groupId, 'use_separate_process') == 1
) {
$this->_shell->execute(
$phpPath . ' %s cron:run --group=' . $groupId . ' --' . Cli::INPUT_KEY_BOOTSTRAP . '='
'%s %s cron:run --group=%s --' . Cli::INPUT_KEY_BOOTSTRAP . '='
. self::STANDALONE_PROCESS_STARTED . '=1',
[
BP . '/bin/magento'
$phpPath,
BP . '/bin/magento',
$groupId,
]
);
continue;
Expand Down Expand Up @@ -848,7 +850,7 @@ private function processPendingJobs(string $groupId, array $jobsRoot, int $curre
/** @var Schedule $schedule */
foreach ($pendingJobs as $schedule) {
if (isset($processedJobs[$schedule->getJobCode()])) {
// process only on job per run
// process only one of each job per run
continue;
}
$jobConfig = isset($jobsRoot[$schedule->getJobCode()]) ? $jobsRoot[$schedule->getJobCode()] : null;
Expand Down
47 changes: 47 additions & 0 deletions app/code/Magento/Cron/Shell/CommandRendererBackground.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
<?php

/**
* Copyright © Magento, Inc. All rights reserved.
* See COPYING.txt for license details.
*/

declare(strict_types=1);

namespace Magento\Cron\Shell;

use Magento\Framework\App\Filesystem\DirectoryList;
use Magento\Framework\Filesystem;
use Magento\Framework\OsInfo;
use Magento\Framework\Shell\CommandRenderer;

class CommandRendererBackground extends CommandRenderer
{
/**
* @param Filesystem $filesystem
* @param OsInfo $osInfo
*/
public function __construct(
private readonly Filesystem $filesystem,
private readonly OsInfo $osInfo,
) {
}

/**
* @inheritDoc
*/
public function render($command, array $arguments = []): string
{
$command = parent::render($command, $arguments);

$logFile = '/dev/null';
if ($groupId = $arguments[2] ?? null) {
$logDir = $this->filesystem->getDirectoryRead(DirectoryList::LOG)->getAbsolutePath();
// @phpcs:ignore Magento2.Functions.DiscouragedFunction.Discouraged
$logFile = escapeshellarg($logDir . 'magento.cron.' . $groupId . '.log');
}

return $this->osInfo->isWindows() ?
'start /B "magento background task" ' . $command
: str_replace('2>&1', ">> $logFile 2>&1 &", $command);
}
}
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
<?php

/**
* Copyright © Magento, Inc. All rights reserved.
* See COPYING.txt for license details.
*/

declare(strict_types=1);

namespace Magento\Cron\Test\Unit\Console\Command;
Expand Down Expand Up @@ -80,7 +82,7 @@ public function testExecute()
new CronCommand($this->objectManagerFactory, $this->deploymentConfigMock)
);
$commandTester->execute([]);
$expectedMsg = 'Ran jobs by schedule.' . PHP_EOL;
$expectedMsg = '';
$this->assertEquals($expectedMsg, $commandTester->getDisplay());
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
<?php

/**
* Copyright © Magento, Inc. All rights reserved.
* See COPYING.txt for license details.
*/

declare(strict_types=1);

namespace Magento\Cron\Test\Unit\Shell;

use Magento\Cron\Shell\CommandRendererBackground;
use Magento\Framework\Filesystem;
use Magento\Framework\Filesystem\Directory\ReadInterface;
use Magento\Framework\OsInfo;
use PHPUnit\Framework\MockObject\MockObject;
use PHPUnit\Framework\TestCase;

/**
* @covers \Magento\Cron\Shell\CommandRendererBackground
*/
class CommandRendererBackgroundTest extends TestCase
{
/**
* Test path to Magento's var/log directory
*
* @var string
*/
protected $logPath = '/path/to/magento/var/log/';

/**
* Test data for command
*
* @var string
*/
protected $testCommand = 'php -r test.php';

/**
* @var Filesystem|MockObject
*/
protected $filesystem;

/**
* @var OsInfo|MockObject
*/
protected $osInfo;

protected function setUp(): void
{
$this->osInfo = $this->getMockBuilder(OsInfo::class)
->getMock();

$directoryMock = $this->getMockBuilder(ReadInterface::class)
->getMock();
$directoryMock->expects($this->any())
->method('getAbsolutePath')
->willReturn($this->logPath);

$this->filesystem = $this->getMockBuilder(Filesystem::class)
->disableOriginalConstructor()
->getMock();
$this->filesystem->expects($this->any())
->method('getDirectoryRead')
->willReturn($directoryMock);
}

/**
* @covers ::render
* @dataProvider commandPerOsTypeDataProvider
*
* @param bool $isWindows
* @param string $expectedResults
* @param string[] $arguments
*/
public function testRender($isWindows, $expectedResults, $arguments)
{
$this->osInfo->expects($this->once())
->method('isWindows')
->willReturn($isWindows);

$commandRenderer = new CommandRendererBackground($this->filesystem, $this->osInfo);
$this->assertEquals(
$expectedResults,
$commandRenderer->render($this->testCommand, $arguments)
);
}

/**
* Data provider for each os type
*
* @return array
*/
public function commandPerOsTypeDataProvider()
{
return [
'windows' => [
true,
'start /B "magento background task" ' . $this->testCommand . ' 2>&1',
[],
],
'unix-without-group-name' => [
false,
$this->testCommand . ' >> /dev/null 2>&1 &',
[],
],
'unix-with-group-name' => [
false,
$this->testCommand . " >> '{$this->logPath}magento.cron.group-name.log' 2>&1 &",
['php-executable', 'script-path', 'group-name'],
],
];
}
}
9 changes: 4 additions & 5 deletions app/code/Magento/Cron/etc/di.xml
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,14 @@
</argument>
</arguments>
</virtualType>
<!-- @api -->
<virtualType name="shellBackground" type="Magento\Framework\Shell">
<virtualType name="shellBackgroundCron" type="Magento\Framework\Shell">
<arguments>
<argument name="commandRenderer" xsi:type="object">Magento\Framework\Shell\CommandRendererBackground</argument>
<argument name="commandRenderer" xsi:type="object">Magento\Cron\Shell\CommandRendererBackground</argument>
</arguments>
</virtualType>
<type name="Magento\Cron\Observer\ProcessCronQueueObserver">
<arguments>
<argument name="shell" xsi:type="object">shellBackground</argument>
<argument name="shell" xsi:type="object">shellBackgroundCron</argument>
<argument name="logger" xsi:type="object">Magento\Cron\Model\VirtualLogger</argument>
</arguments>
</type>
Expand Down Expand Up @@ -65,7 +64,7 @@
<arguments>
<argument name="tasks" xsi:type="array">
<item name="cronMagento" xsi:type="array">
<item name="command" xsi:type="string">{magentoRoot}bin/magento cron:run | grep -v "Ran jobs by schedule" >> {magentoLog}magento.cron.log</item>
<item name="command" xsi:type="string">{magentoRoot}bin/magento cron:run >> {magentoLog}magento.cron.log 2>&amp;1</item>
<item name="optional" xsi:type="boolean">false</item>
</item>
</argument>
Expand Down
7 changes: 7 additions & 0 deletions app/etc/di.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2006,4 +2006,11 @@
<preference for="Magento\Framework\Filter\Input\PurifierInterface" type="Magento\Framework\Filter\Input\Purifier"/>
<preference for="Magento\Framework\App\PageCache\IdentifierInterface" type="Magento\Framework\App\PageCache\Identifier"/>
<preference for="Magento\Framework\App\State\ReloadProcessorInterface" type="Magento\Framework\App\State\ReloadProcessorComposite" />

<!-- @api -->
<virtualType name="shellBackground" type="Magento\Framework\Shell">
<arguments>
<argument name="commandRenderer" xsi:type="object">Magento\Framework\Shell\CommandRendererBackground</argument>
</arguments>
</virtualType>
</config>