Monday, February 5, 2018

TYPO3 - How to render a Fluid standalone view multiple times in different languages

Back in 2015, I wrote 2 blogposts (first and second) about rendering a localized Fluid standalone view in a scheduler task (commandController). The main problem was to render a Fluid standalone view multiple times within the same request but with different languages. Back then, my solution was to create an own ViewHelper and a modified version of the TYPO3 LocalizationUtility which were responsible for handling the localization changes during the rendering request.

Meanwhile, I got feedback from readers of my blog pointing me to a more simple solution for the problem.

The main problem with changing the TYPO3 backend language during one request is the language cache, which is only initialized once for the current language. So when you switch the backend language multiple times, the cached language files for the previous language will still be used.

The solution is to unset the language cache for the extension you are rendering your Fluid standalone view from. In order to do so, you have to extend the TYPO3 LocalizationUtility with a new function as shown below.

class LocalizationUtility extends \TYPO3\CMS\Extbase\Utility\LocalizationUtility
     * Resets the language cache for the given extension key
     * @param string $extensionName
    public static function resetLocalizationCache($extensionName)

The example code below shows, how to use the resetLocalizationCache method before rendering a Fluid standalone view in a given language.

 * Renders a Fluid StandaloneView respecting the given language
 * @param string $language The language (e.g. de, dk or se)
 * @return string
public function renderStandaloneView($language = '')
    // Set the extensionKey
    $extensionKey = GeneralUtility::underscoredToUpperCamelCase('standaloneview');

    if ($language !== '') {
        // Temporary set Language of current BE user to given language
        $GLOBALS['BE_USER']->uc['lang'] = $language;

    /** @var \TYPO3\CMS\Fluid\View\StandaloneView $view */
    $view = $this->objectManager->get(StandaloneView::class);
    $template = GeneralUtility::getFileAbsFileName(

    // Set Extension name, so localizations for extension get respected

    return $view->render();

So with a small extension of the LocalizationUtility it is now easily possible to render a Fluid standalone view in a language of choice and it is also possible to switch the language within the same request (e.g. sending out localized e-mails to multiple recipients).

I updated the Demo-Extension, which contains a backend module and a command controller for demonstration purposes.

I would like to thank Johannes Rebhan for giving me the hint about the localization cache and also Ulrich Fischer for showing me a different approach, which requires more code, but lead to the same result.

Thursday, January 25, 2018

SSH reports "Too many Authentication Failures" on first connect

Today I wanted to connect to a new clients SSH server and received a "Too many Authentication Failures" message just on the first connect to the host. After a short break and some Google research, I found the very simple reason for the message.

Since I have several SSH keys in my .ssh/ directory, SSH tries to use each of it to connect to the SSH server. So when the SSH server has a very low "MaxAuthTries" setting configured, then the SSH connection may fail before password authentication is offered.

In order to connect a SSH servers with a low "MaxAuthTries" setting, you can use the following command:

ssh -o PubkeyAuthentication=no [email protected]

After using the "PubkeyAuthentication=no" option, I could login to the host and add a SSH public key to the .ssh/authorized_keys file.

Wednesday, June 28, 2017

TYPO3 8.7 LTS - How to enable the image cropping tool in your own extension

People using TYPO3 7.6 or 8.7 for sure know the cool Image Cropping tool that was introduced with TYPO3 7. As an extension developer, you can easily switch on the Image Cropping tool for your own extension by enabling adding the imageoverlayPalette to the foreign_types config array for the FILETYPE_IMAGE in the TCA like shown in this blogpost by Marcus Schwemer. In TYPO3 8.7 the Image Cropping tool does not show as expected, due to structural TCA changes in the TYPO3 8.

For TYPO3 8.7, the following (minimal) TCA configuration will enable the Image Cropping tool for the field "image" in an own extension:

'image' => [
    'exclude' => 1,
    'label' => 'LLL:EXT:custom_extension/Resources/Private/Language/locallang_db.xlf:tx_customextension_domain_model_tablename.custom_field',
    'config' => \TYPO3\CMS\Core\Utility\ExtensionManagementUtility::getFileFieldTCAConfig('image', [
        'foreign_match_fields' => [
            'fieldname' => 'image',
            'tablenames' => 'tx_customextension_domain_model_tablename',
            'table_local' => 'sys_file',
        'overrideChildTca' => [
            'types' => [
                \TYPO3\CMS\Core\Resource\File::FILETYPE_IMAGE => [
                    'showitem' => '
        'minitems' => 0,
        'maxitems' => 999,
    ], $GLOBALS['TYPO3_CONF_VARS']['GFX']['imagefile_ext']),

Note, that the configuration array only shows the configuration for a single field and that the array must be located in the "columns"-array of your domain model TCA. For TYPO3 8.7, the overrideChildTca config is the important part, which enables the Image Cropping tool.

If your extension needs to support TYPO3 7.6 and 8.7, your TCA should include both the overrideChildTca config as described in this blogpost and the foreign_types config as described in Marcus's blogpost.

Monday, June 26, 2017

TYPO3 Extbase - Manual validation of a domain model

When you create an Extbase extension and allow a website user to submit form data that will be saved to the TYPO3 database, you usually work with validators in your domain model to ensure, that the given data will match certain criteria (e.g. properties have the right data type or expected data format). Extbase offers an easy way to add validation rules to domain model properties, by just adding the @validate annotation followed by one or multiple validators as shown in the following example:

 * Description
 * @var string
 * @validate NotEmpty, StringLength(minimum=10, maximum=50)
protected $description = '';

Extbase will take care of the domain model validation, when the given form data is converted to an object of the type TYPO3\CMS\Extbase\Mvc\Controller\Argument.

Manual validation

If you manually create a domain model object and want to make sure, that your Extbase validation rules are meet, you can trigger the domain model validation manually as shown below:

/** @var Data $dataModel */
$dataModel = $this->objectManager->get(Data::class);
$dataModel->setDescription('too short');

/* @var $validator \TYPO3\CMS\Extbase\Validation\Validator\ConjunctionValidator */
$validator = $this->objectManager->get(ValidatorResolver::class)->getBaseValidatorConjunction(Data::class);
$validationResults = $validator->validate($dataModel);

if ($validationResults->hasErrors()) {
    // @todo cycle through errors in $validationResults->getFlattenedErrors()

By creating the domain model object manually, you must take into account, that this will create a new object, where all properties are initialized with the default values defined in the domain model.

Practical use case

As an example for a practical use case for manual domain model validation, lets assume you have a REST webservice and need to import some data to TYPO3. You typically fetch the data from the webservice and add the data to the database. Instead of checking the content of the incoming record/field manually using if-statements, you can use the Extbase property mapper in combination with manual domain model validation.

Below follows some example code for the described use case:

/** @var PropertyMapper $propertyMapper */
$propertyMapper = $this->objectManager->get(PropertyMapper::class);

// Get some data - could for example be some data from a REST webservice
$data = $this->getApiData();

foreach ($data as $importRecord) {
    $dataModel = $propertyMapper->convert($importRecord, Data::class);
    // Note: The propertyMapper will set domain model default values for all all non-mappable values

    /* @var $validator \TYPO3\CMS\Extbase\Validation\Validator\ConjunctionValidator */
    $validator = $this->objectManager->get(ValidatorResolver::class)->getBaseValidatorConjunction(Data::class);
    $validationResults = $validator->validate($dataModel);

    if ($validationResults->hasErrors()) {
        // Record could not be imported, collect error messages for each field in $errorMessages array

        /** @var Error $error */
        foreach ($validationResults->getFlattenedErrors() as $field => $errors) {
            $errorMessages = [];
            foreach ($errors as $error) {
                $errorMessages[] = $error->getMessage();
    } else {
        // Import record to repository...

By using the Extbase property mapper to create domain model objects, you do not need to check and assign each field individually. You just have to make sure, that the array given passed to the "convert" function use the same field naming as the domain model do like shown below.

'title' => 'a title',
'email' => '[email protected]',
'description' => 'a description',
'year' => 2017,
'amount' => 19.99,
'paid' => true

Note, that the resulting object from the property mapper will contain the default values for each property, that can't be mapped properly.

In the code example above, the resulting domain model object will manually be validated and if no validation errors occur, the object can be added to the repository.

I created a small demo extension with a command controller, which contains 2 example commands that show validation results for some dummy data.

Monday, May 15, 2017

How to use MySQL FIELD-function in TYPO3 8.7 with Doctrine DBAL

When you want to query an Extbase repository for a list of UIDs or PIDs, you usually add a query constraint like query->in('pid', $pidList) to the query, where $pidList is an array of integers. But what, if you want to control the sorting of the returned records? Lets assume, you want to select the following list of UIDs [5, 3, 4, 1] from your repository and the sorting or the UIDs must remain.
Extbase only has the possibility to sort the query result by a given column either ascending or descending, so there is no possibility to control so returned sorting as intended.

In order to resolve the problem, I found this very helpful article from Manfred Rutschmann, where he describes exactly the same situation and offers a solution for the problem, which works great in TYPO3 7.6. Sadly the solution does not work in TYPO3 8.7 LTS and fails with an exception, that e.g. column tx_myext_domain_model_mymodel.uid=5 does not exist.

Since I'm using MySQL as a database engine, I tried to find a solution, where I could use the MySQL FIELD-function to apply the special sorting to the ORDER BY clause.

Fortunately Doctrine DBAL was integrated in TYPO3 8.7 LTS, which offers an easy and readable way to construct my query. I added the following function to my Extbase repository.

In line 31 I add my own ORDER BY FIELD clause, where the special sorting is applied

Since the queryBuilder just returns an array of database records (where each records is just an array of returned fields/values), I use the TYPO3 DataMapper in line 35 to map the returned rows as objects, so the returned QueryResult object will contain objects of the type Mymodel

The example shown has one little downside, since it only works with a MySQL Database.

If you want to get more details about the Doctrine DBAL integration in TYPO3, make sure to read the documentation.

Wednesday, January 25, 2017

TYPO3 - Adding direct mail fields to femanager

In a project I needed to add the direct mail fields "Activate Newsletter" and "Subscribe to categories" to the TYPO3 femanager (thanks to Alex Kellner for this great extension), so frontend users are able to subscribe to a newsletter and select newsletter categories.

Since femanager is created with Extbase, it is easily extendable and I created a small TYPO3 extension named femanager_dmail_subscribe, that automatically adds "Newsletter subscription", "Newsletter category" and "HTML newsletter" direct mail fields to femanager, so editors can select the direct mail fields a frontend user should be able to edit.

femanager plugin field settings

In order to display direct mail categories in the frontend, it is required to add the sysfolder with the direct mail categories to the femanager record storage page.

femanager plugin record storage page
After installing and configuring the extension as shown above, frontend users can edit the selected fields (see screenshot below)

Frontend user can edit direct mail fields

I uploaded 2 versions of the extension to TER. Version 1.0.0 is compatible with TYPO3 6.2 and femanager 1.5.2 and version 2.0.0 is compatible with TYPO3 7.6 and femanager 2.x

Tuesday, October 4, 2016

Testing and deploying TYPO3 using GitLab, GitLab CI, Capistrano and TYPO3 Console

In this article, I'll show you how to setup automated testing and continuous deployment of a TYPO3 website using GitLab, GitLab CI, Capistrano and TYPO3 Console. The main purpose of the article is to give you a starting point for automated testing and continuous deployment.

With the setup described in this article, I want to make sure, that
  • tests are executed automatically on every commit
  • pushes to master branch will automatically be deployed to the production server
  • deployment will only execute, when tests pass

I tried to keep write the article as simple as possible, so please note, that some features / processes of real life automated deployment scenarios are not covered.

  • A composer based TYPO3 website with a basic site package
  • At least one TYPO3 extension with tests
  • A working GitLab server (I'm using an own instance of the Community Edition)
  • A GitLab CI runner
  • A SSH account on the webspace, where you want the TYPO3 website to be deployed
  • Ruby version 1.9.3 or newer on your local machine (required for capistrano) offers free private repositories and also access to GitLab CI runners, so I'm pretty sure, that the setup I described here may even work using the infrastructure.

1. Structure of the composer based TYPO3 website

The TYPO3 website used for this blogpost is a really simple composer based TYPO3 website. It contains a sitepackage with TypoScript and a Fluid template and it contains the TYPO3 extension sf_yubikey, which is used in this example to execute some unit tests. Also the TYPO3 website contains the TYPO3 extension typo3_console, which will be used in the deployment process. 

Unit tests can be executed on CLI with the command shown below:
./vendor/bin/phpunit -c web/typo3conf/ext/sf_yubikey/Tests/Build/UnitTests.xml

My composer.json file contains the following contents:
 "repositories": [
  { "type": "composer", "url": "" }
 "name": "typo3/cms-base-distribution",
 "description" : "TYPO3 CMS Base Distribution",
 "license": "GPL-2.0+",
 "require": {
  "typo3/cms": "^7.6",
  "typo3-ter/sf-yubikey": "*",
  "helhum/typo3-console": "^3.3"
 "require-dev": {
   "mikey179/vfsStream": "1.4.*@dev",
   "phpunit/phpunit": "~4.7.0"
 "extra": {
  "typo3/cms": {
   "cms-package-dir": "{$vendor-dir}/typo3/cms",
   "web-dir": "web"
  "helhum/typo3-console": {
   "install-binary": false

Note, that I use helhum/typo3-console (not from TER), which on install makes the typo3cms command available in the vendor/bin directory.

The TYPO3 installation also contains a backend user named _cli_lowlevel, which is required to execute CLI commands.

2. Move all sensitive data and local settings out of the LocalConfiguration.php

You should never commit any sensitive data (like passwords or API keys) to a Git repository, so I move those settings out of from LocalConfiguration.php to the file AdditionalConfiguration.php, which also is located in the typo3conf/ folder.

The local AdditionalConfiguration.php file used in this blogpost looks like shown below:

$GLOBALS['TYPO3_CONF_VARS']['DB']['database'] = 'typo3db';
$GLOBALS['TYPO3_CONF_VARS']['DB']['host']     = 'localhost';
$GLOBALS['TYPO3_CONF_VARS']['DB']['username'] = 'dbusername';
$GLOBALS['TYPO3_CONF_VARS']['DB']['password'] = 'dbpassword';

$GLOBALS['TYPO3_CONF_VARS']['BE']['installToolPassword'] = 'hashvalue';

$GLOBALS['TYPO3_CONF_VARS']['GFX']['im_path'] = '/opt/local/bin/';
$GLOBALS['TYPO3_CONF_VARS']['GFX']['im_path_lzw'] = '/opt/local/bin/';

Besides all sensitive data from the LocalConfiguration.php, the AdditionalConfiguration.php file should include all settings, that should not be shared across deployment stages.

3. Modify the default .gitignore, so some TYPO3 directories will not be included in the git repository

For the TYPO3 website in this blogpost, I extended the default .gitignore file, so contents of some TYPO3 directories and files do not get committed to the git repository (e.g. fileadmin, uploads or all extensions installed by composer)

# TYPO3 CMS Base Distribution

# Ignore fileadmin, uploads and typo3temp

# Ignore some files in typo3conf

# Ignore language files (fetched by TYPO3 console)

# Ignore all extensions (loaded by composer)

# But include the sitepackage, which is not composer based

# Ignore capistrano deployment logs

4. Commit the TYPO3 website to a git repository

The .gitignore file is ready, so now I create a new git repository and add / commit all files to it. Finally, I create a new project on the GitLab server and add the repository as a remote for the local git repository and finally push the repository to the remote.

git remote add origin [email protected]:derhansen/typo3_ci_cd.git
git push -u origin master

5. Configure deployment of TYPO3 website with capistrano

5.1 SSH setup for deployment to webserver

First, I add my public SSH RSA key to the .ssh/authorized_keys file in the home-directory of the SSH user on the webserver, so I'm able to login by using SSH key authentication. This step is not required, but makes deployment from my local machine to the webserver easier.

Next I create a new SSH RSA key on the webserver (like shown here), so the SSH user will be able to pull from the remote git repository on the GitLab server I created in step 4. This SSH RSA Key will be the deployment key and must not contain a password.

The new SSH public key must now be added to the git repository on the GitLab server. In order to do so, I add it as Deployment Key for the project (see screenshot below)

5.2 Initialize and configure capistrano

I have chosen capistrano as the deployment tool for the TYPO3 website, because it is well documented, has a large user base and is easy to use/extend. If you do not like capistrano, it should be easy to replace capistrano with the deployment tool of your choice (e.g. TYPO3 surf, Deployer, Magallanes, ...)

In the project root of my local composer based TYPO3 website, I now initialize a new capistrano config with the following command:

cap install

First of all, I remove everything in the file config/deploy/production.rb and add the following content:

server 'typo3-ci-cd.domain.tld',
  user: 'typo3cicd',
  roles: %w{app db web}

Next, I create a really basic deployment in config/deploy.rb, which mainly executes composer install and executes some TYPO3 tasks using TYPO3 console.

# config valid only for current version of Capistrano
lock '3.6.1'

set :application, 'typo3_ci_cd'
set :repo_url, '[email protected]:torben/typo3_ci_cd.git'
set :deploy_to, '/var/www/typo3-ci-cd/httpdocs'
set :scm, :git
set :keep_releases, 5
set :linked_dirs, %w{web/fileadmin web/uploads}
set :linked_files, %w{web/typo3conf/AdditionalConfiguration.php}

namespace :composer do
    desc "Runs composer."
    task :install do
        on roles(:web) do
            within release_path do
                execute "composer", "install", "--no-dev", "--no-interaction", "--prefer-dist"

namespace :typo3 do

    desc "Clear TYPO3 Cache"
    task :cache_flush do
        on roles(:all) do
            within release_path do
                execute "#{release_path}/vendor/bin/typo3cms cache:flush"

    desc "Update language"
    task :language_update do
        on roles(:all) do
            within release_path do
                execute "#{release_path}/vendor/bin/typo3cms language:update"


namespace :deploy do
    after :publishing, "composer:install"
    after :finishing,  "typo3:cache_flush"
    after :finishing,  "typo3:language_update"

You may note the file AdditionalConfiguration.php in the linked_files section of the deployment configuration. This is required in order to get TYPO3 running, when I'm doing a deployment from my local machine.

The linked_dirs section contains the folders web/fileadmin and web/uploads, which I want to share across deployments, since users will upload files to it.

5.3 Manual steps before first deployment

Before the first deployment can be executed, there are some steps, that need to be done manually on the webserver.

5.3.1 Initial shared folder setup

I copy the contents of the directories fileadmin/ and uploads/ from my local TYPO3 website to the directory shared/web in the deployment path (see "deploy_to" in the deploy.rb file). The final directory structure is as shown below (Note: .htaccess files are not shown).

└── web
    ├── fileadmin
    │   ├── _temp_
    │   │   └── index.html
    │   └── user_upload
    │       ├── index.html
    │       └── _temp_
    │           ├── importexport
    │           │   └── index.html
    │           └── index.html
    ├── typo3conf
    │   └── AdditionalConfiguration.php
    └── uploads
        ├── index.html
        ├── media
        │   └── index.html
        ├── pics
        │   └── index.html
        └── tx_felogin

The shared folder does also contains the typo3conf/ directory, where I add the AdditionalConfiguration.php and modify the settings (e.g. database credentials, GraphicsMagick path, ...) to match the server configuration.

5.3.2 Initial database setup

Since there is no initial TYPO3 database on the webserver, I create a local dump of the TYPO3 database and restore it to the remote database on the webserver.

5.4 Initial deployment

Now it is time to do the first deployment form my local machine, so I execute the deployment with the following command

cap production deploy

Capistrano now executes the deployment and after finishing, I have a fully working copy of my local TYPO3 website on the remote webserver.

Capistrano creates the directories current/, releases/, repo/ and shared/. The current version of the deployed TYPO3 website is available in the current/ directory, so I adjusted the webserver to load the website from that directory.

5.5 Commit deployment configuration

Finally I commit the deployment configuration to the git repository, so it later can be used for the automated deployment.

6. Configure automated test execution on GitLab server

In an automated deployment process, you need to make sure, that only working versions of your website will be deployed to the production server. In this example, I simply run the unit tests of a TYPO3 extension to demonstrate, how automatic test execution can be a part of a deployment process. In real life scenarios, you can for example use unit, functional or acceptance tests to make sure, that your website will run smoothly after deployment.

In my local project, I create the file .gitlab-ci.yml and add the following contents to it:

  - tests

  stage: tests
  image: php:7.0
    - apt-get update -yqq
    - apt-get install git libcurl4-gnutls-dev libicu-dev libvpx-dev libjpeg-dev libpng-dev libxpm-dev zlib1g-dev libfreetype6-dev libxml2-dev libexpat1-dev libbz2-dev libgmp3-dev libldap2-dev unixodbc-dev libpq-dev libsqlite3-dev libaspell-dev libsnmp-dev libpcre3-dev libtidy-dev -yqq
    - docker-php-ext-install mbstring zip
    - curl -sS | php
    - php composer.phar install
    - php vendor/bin/phpunit --colors -c web/typo3conf/ext/sf_yubikey/Tests/Build/UnitTests.xml

I add and commit the file to the git repository and push the changes to the remote. Now GitLab CI will start a PHP 7.0 docker container, install all dependencies and finally run the configured unit tests.

Nice, the build passed....

...and unit tests have been executed on the GitLab CI runner.

You may notice, that the build took 3:16 minutes to finish, which is pretty long just for unit test execution. To reduce the build time, I suggest to create your own docker image with all dependencies included.

If you want to receive e-mail notifications on builds (or only for failed builds), you can enable this feature in the service section of the GitLab project.

7. Configure automated deployment on GitLab server

As a final step, I want to automate the deployment of the website, when commits are made to the master branch and if tests passed. Since the GitLab CI runner will do the deployment, it must be able to SSH to the production website.

I create a new SSH RSA key (with no password), which will be used for the deployment from the GitLab CI runner to the production webserver. After adding the SSH public key to the ssh/authorized_keys file of the SSH user on the webserver, I create a new variable in the GitLab project and add the new private key to it.

Next, I extend the .gitlab-ci.yml to use the SSH_PRIVATE_KEY variable in the docker container, that will do the deployment. Detailed instructions can be found in the GitLab CI documentation. The final configuration file looks like shown below:

  - test
  - deploy

  stage: test
  image: php:7.0
    - apt-get update -yqq
    - apt-get install git libcurl4-gnutls-dev libicu-dev libvpx-dev libjpeg-dev libpng-dev libxpm-dev zlib1g-dev libfreetype6-dev libxml2-dev libexpat1-dev libbz2-dev libgmp3-dev libldap2-dev unixodbc-dev libpq-dev libsqlite3-dev libaspell-dev libsnmp-dev libpcre3-dev libtidy-dev -yqq
    - docker-php-ext-install mbstring zip
    - curl -sS | php
    - php composer.phar install
    - php vendor/bin/phpunit --colors -c web/typo3conf/ext/sf_yubikey/Tests/Build/UnitTests.xml

  stage: deploy
  image: ruby:2.3
    - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
    - eval $(ssh-agent -s)
    - ssh-add <(echo "$SSH_PRIVATE_KEY")
    - mkdir -p ~/.ssh
    - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
    - gem install capistrano
    - cap production deploy
    - master

Since capistrano is a ruby application, I use a docker container with ruby for the deployment.

Finally, after committing the changes to the repository, the GitLab CI runner runs the build and deploys the TYPO3 website to the production server.

And that's it, the automated testing and continuous deployment setup for TYPO3 is up and running.

Outlook and conclusion

There are for sure some things that can be improved in the process I described in this post. For example, you could also add the AdditionalConfiguration.php file as a secret variable to the GitLab project and use the GitLab CI Mutli-Runner to deploy it to the production server. Also you can automate TYPO3 related tasks using TYPO3 console (e.g. database updates on new extension installation with typo3cms database:updateschema, update reference index, ...).

I hope the post will give you a good point of how/where to start with automating tests and continuous deployment of a TYPO3 website. The shown technique is just one way of how to setup and automate the process - feel free to build your own configuration with the tools of your choice and please share your knowledge :-)