A transactional notification is a notification that is intended only for the end-user and cannot be sent to anyone else. This might be because it contains sensitive information, such as a password reset link, or details of their account.
Allowing transactional notifications to be sent to anyone could be a serious security risk for your WordPress website and as such, the amount of customisation that can be done is quite limited. This is partially because of how WordPress works and partially how I feel transactional notifications should be treated for the benefit of user security on sites with Better Notifications for WordPress installed.
Please see the Types of Notifications available in Better Notifications for WordPress for a list of Transactional emails that are available to customise.