Response to consultation Paper on draft RTS and ITS on benchmarking portfolios
Go back
The proposed benchmarks feature major weaknesses; hence, as benchmarks, they are not fit for purpose. Good benchmarks need to fulfil a number of minimum criteria, among them (i) clarity of definition and uniqueness, and (ii) stability in time over several benchmarking cycles, at least.
RTS Art. 3 (2) a) (assessment of extreme values) violates criterion (i) unless the absolute maximum and minimum of the sample are implied by the term “extremes”. RTS Art. 3 (2) b) (first and fourth quartiles) violates criterion (ii) in that the same number of portfolios (resp. models, resp. banks) will be subject to increased scrutiny and specific supervisory assessment in every benchmarking cycle, no matter what changes were implemented by banks as a result of the preceding benchmarking cycles.
To solve these issues, we propose two changes to the RTS Art. 3 (2a) and (2b):
• RTS Art. 3(2) a): Replace the ill-defined term “extremes” by “outliers”. This term is defined in statistics, and there are easy operational procedures in descriptive statistics to detect evidence for outliers (cf. below). Moreover, we are convinced that an outlier – a data point a significant distance away from the body of a sample distribution – is a valid target for an assessment by supervisors, indeed.
• RTS Art. 3(2) b): Replace the metric of first and fourth quartiles by the metric of outliers as identified by box plots. A box plot will display stability in time when most of the changes induced by the benchmarking procedure are in the outlier portfolios but will adjust moderately when a large fraction of the portfolios evolve in time. Additionally, we would very much appreciate a definition of an "acceptable level of variance". Only variances beyond this acceptable level of variance should be examined further.
Specifically, in the standard implementation of Box plots,
- a uniform distribution displays no outliers,
- a normal distribution displays rare outliers on both sides,
- a symmetric fat-tailed distribution displays frequent outliers on both sides, and
- a skewed distribution displays frequent outliers on one side only.
Most importantly, however, we believe that the metrics proposed in the RTS are inconsistent with Art. 78 (5) of the Directive 2013/36/EU, due to the fact that the proposed benchmarks indicate a putative need for action that is de facto unwarranted meaning that supervisory authorities are urged to take actions that lead to a standardisation of the approaches thus creating wrong incentives and triggering herd behaviour."
Under the current proposal, 50% of the internal approaches would automatically be conspicuous. In our view, this is neither meaningfulnor appropriate. Hence, we suggest using an outlier identification that is based on box plots (c.f. our above answer under Q2). At this juncture, the definition of an acceptable level of variance" is important. One alternative "second-best" regulatory option the EBA might wish to consider is a reduction of the review scope resulting from changed quantile rules. From our point of view the use of the 90% and 10% quantiles is rather more productive for the mentioned purposes.
Implicitly, the approach is based on the working hypothesis that the median corresponds to the "true level of risk". However, this is by no means correct (c.f. our presentations under the section entitled General Comments).
(ii) comparison between own funds under the internal models and the standardised approach
The calculation of a standardised approach is a laborious and costly procedure and should be avoided. The reasons are manifold, but in particular that using the standardised approach as a benchmark presumes that it is related to the “true level of risk“. However, as it is a static measure it is not reacting to volatile markets, as – for example – the market risk models do. Moreover, banks using an approved market risk model for calculating RWAs usually do not have an implementation for calculating the standardised approach – and are not required to.
Furthermore, due to their insufficient risk sensitivity particularly in the credit risk area of the CRSA, standardised approaches cannot be regarded as an appropriate benchmark. More often than not, standardised approaches either underestimate or overestimate the actual risk. This, too, renders them inappropriate as benchmarks. Using standardised approaches as a benchmark during the variance analysis between the standardised approach and the internal approaches would merely illustrate the shortcomings of the standardised approaches. Due to their lack of risk sensitivity, the use of standardised approaches promotes herd behaviour and creates wrong incentives (i.e. incentives which are not risk sensitive).
Furthermore, for banks using internal approaches, an obligation is introduced through the "backdoor" to implement the standardised approach. This is inconsistent with the current CRR rules; as a result, the draft RTS exceed their legal mandate in an inappropriate manner. We have extremely strong reservations over this fact. Unless this benchmark is abandoned in light of the above caveats, we assume that the supervisor will either perform the RWA calculation using the standardised approaches or that the supervisor will outsource the latter to a provider of reporting software.
(iii) comparison between estimates and outturns
First of all, EBA has to ensure that all participating IRBA-institutes use the same definitions of default. Furthermore it is obvious that a modification of the consideration of collaterals would lead to a change in Loss Given Default. Failure to deal with those two issues would lead to distorted results.
Calculating the RWA on historical defaults and losses involves two problems:
• Low default rates in wholesale portfolios lead to the problem that quite a lot rating classes do not contain a default (depending on the size of portfolio).
This is especially problematic for the better ratings classes. Consequently an aggregation of rating classes is necessary to receive reasonable results.
• To ensure a homogeneous determination of loss rates, there is a need for an EBA standard. Actually there is no specification given. Especially in the case of the one year horizon a determination of a realised loss is quite impossible because the duration of realisation is longer than the given horizon.
In our view, the use of "outturns" versus "estimates" gives rise to a host of methodological problems. For instance, already the number of the rating classes used in the IRBA might change the confidence intervals - and thus potentially a bank's conspicuousness. This may lead to wrong conclusions."
For the IRBA, it may be helpful to use estimates offered by pool data operators as a benchmark. In Germany, there are several such providers.
Furthermore, we would welcome a limitation to plain vanilla instruments. This enhances comparability given that not all banks are able (and, what is more, do not have to be able) to perform a valuation of e.g. double-no-touch options and variance swaps. However, according to Annex VII a, this exercise appears to be clearly more comprehensive than the comparative computation by the Trading Book Group. There is a need to calculate more instruments in more portfolios featuring a greater number of combinations which clearly drives up the operational costs (instrument setup, portfolio setup and portfolio statement, the first two of which are partly purely manual). As far as cost-benefit aspects are concerned, less is clearly more at this point (in the absence of any sacrifice in terms of added insights).
One major advantage in our view is the option of an analysis at the level of risk factors. Prior to the actual exercise, the test of the portfolios should be performed in close cooperation with the banks affected. This way, potential inconsistencies and misunderstandings can be addressed in advance. Also, a Q&A process should be established.
• The assessment criterion comparison to the standard approach" should be deleted (for a more detailed explanation cf. reply to Q3)
• The impact of the quartile-based assessment criterion should be mitigated by changed quantiles (e.g. 10%). Cf. also the answer to Q2 (box plots)."
Please confirm that the counterparties are to be understood as the legal entities. I.e. if ACME Inc. is listed as counterparty in the sample portfolio, the information requested refers to the legal entity ACME Inc. and not to any other entities / subsidiaries in the group of connected clients to which ACME Inc. belongs.
With regard to the report on the consolidated basis, clarification should be provided on how to report credit risk parameters which are not identical within the banking group. E.g., the same counterparty may have different ratings in different subsidiaries of the banking group or there may be different approaches to quantifying collateral haircuts in different subsidiaries. One possible approach would be to use the parameters of the group company with the largest exposure share for the sample portfolio. This might however result in a mismatch between the one specified PD and the overall reported RWA amount (calculated with the actual diverging PDs).
The portfolio of large corporates is defined as “annual turnover > EUR 200 mln”. Banks’ internal rating models for corporates are likely to have deviating turnover categories. I.e. one specific rating system may contain corporates with a turnover above as well as below EUR 200 mln. For data such as “default rate past 5 years” banks should be allowed to deliver this figure based on the turnover categories of their corporate rating systems, because it would be a laborious additional operating procedure to calculate a 5 year past default rate based on a portfolio definition that does not correspond to the banks’ internal corporate rating system categories.
Hence, banks featuring internal approaches which lack supervisory authorisation should be explicitly excluded from the benchmarking exercise. Missing authorisations have a considerable impact on the degree of divergence (this has been demonstrated e.g. by the results of the SIG-TB). Hence, they would render benchmarking approaches obsolete. Individual banks still lacked a supervisory authorisation for internal approaches concerning the additional default risks and migration risks in the so-called IRC portfolios. These banks considerably overestimate the risks; the latter only became possible given the missing use for the purposes of calculating capital charges.
Individual clusters may be immaterial for individual banks. In such cases, further exemptions should be possible. To this end, the EBA could stipulate a relative materiality threshold (e.g. 5% max. of the total RWA).
Q2. Do you consider that the benchmarks outlined in the RTS are sufficiently proportionate and flexible? Do you have any alternative benchmark proposals? If yes, please provide details.
The or“ link between the four benchmark criteria implies the need for a closer examination of the internal approaches for almost all portfolios in all banks. In our view, given the fact that the benchmarking approach produces only a limited amount of useful information, this is no longer sufficiently proportionate. Also, it is worth pointing out that - in the event of benchmarks which are not fit for purpose - any unusual outcomes shall and may not be used as an indicator for the appropriateness of a bank's risk assessment.The proposed benchmarks feature major weaknesses; hence, as benchmarks, they are not fit for purpose. Good benchmarks need to fulfil a number of minimum criteria, among them (i) clarity of definition and uniqueness, and (ii) stability in time over several benchmarking cycles, at least.
RTS Art. 3 (2) a) (assessment of extreme values) violates criterion (i) unless the absolute maximum and minimum of the sample are implied by the term “extremes”. RTS Art. 3 (2) b) (first and fourth quartiles) violates criterion (ii) in that the same number of portfolios (resp. models, resp. banks) will be subject to increased scrutiny and specific supervisory assessment in every benchmarking cycle, no matter what changes were implemented by banks as a result of the preceding benchmarking cycles.
To solve these issues, we propose two changes to the RTS Art. 3 (2a) and (2b):
• RTS Art. 3(2) a): Replace the ill-defined term “extremes” by “outliers”. This term is defined in statistics, and there are easy operational procedures in descriptive statistics to detect evidence for outliers (cf. below). Moreover, we are convinced that an outlier – a data point a significant distance away from the body of a sample distribution – is a valid target for an assessment by supervisors, indeed.
• RTS Art. 3(2) b): Replace the metric of first and fourth quartiles by the metric of outliers as identified by box plots. A box plot will display stability in time when most of the changes induced by the benchmarking procedure are in the outlier portfolios but will adjust moderately when a large fraction of the portfolios evolve in time. Additionally, we would very much appreciate a definition of an "acceptable level of variance". Only variances beyond this acceptable level of variance should be examined further.
Specifically, in the standard implementation of Box plots,
- a uniform distribution displays no outliers,
- a normal distribution displays rare outliers on both sides,
- a symmetric fat-tailed distribution displays frequent outliers on both sides, and
- a skewed distribution displays frequent outliers on one side only.
Most importantly, however, we believe that the metrics proposed in the RTS are inconsistent with Art. 78 (5) of the Directive 2013/36/EU, due to the fact that the proposed benchmarks indicate a putative need for action that is de facto unwarranted meaning that supervisory authorities are urged to take actions that lead to a standardisation of the approaches thus creating wrong incentives and triggering herd behaviour."
Q3. What limitations do you see in relation to the use of the proposed benchmarks, i.e., (i) first and the fourth quartiles; (ii) comparison between own funds under the internal models and the standardised approach; and (iii) comparison between estimates and outturns?
(i) first and the fourth quartilesUnder the current proposal, 50% of the internal approaches would automatically be conspicuous. In our view, this is neither meaningfulnor appropriate. Hence, we suggest using an outlier identification that is based on box plots (c.f. our above answer under Q2). At this juncture, the definition of an acceptable level of variance" is important. One alternative "second-best" regulatory option the EBA might wish to consider is a reduction of the review scope resulting from changed quantile rules. From our point of view the use of the 90% and 10% quantiles is rather more productive for the mentioned purposes.
Implicitly, the approach is based on the working hypothesis that the median corresponds to the "true level of risk". However, this is by no means correct (c.f. our presentations under the section entitled General Comments).
(ii) comparison between own funds under the internal models and the standardised approach
The calculation of a standardised approach is a laborious and costly procedure and should be avoided. The reasons are manifold, but in particular that using the standardised approach as a benchmark presumes that it is related to the “true level of risk“. However, as it is a static measure it is not reacting to volatile markets, as – for example – the market risk models do. Moreover, banks using an approved market risk model for calculating RWAs usually do not have an implementation for calculating the standardised approach – and are not required to.
Furthermore, due to their insufficient risk sensitivity particularly in the credit risk area of the CRSA, standardised approaches cannot be regarded as an appropriate benchmark. More often than not, standardised approaches either underestimate or overestimate the actual risk. This, too, renders them inappropriate as benchmarks. Using standardised approaches as a benchmark during the variance analysis between the standardised approach and the internal approaches would merely illustrate the shortcomings of the standardised approaches. Due to their lack of risk sensitivity, the use of standardised approaches promotes herd behaviour and creates wrong incentives (i.e. incentives which are not risk sensitive).
Furthermore, for banks using internal approaches, an obligation is introduced through the "backdoor" to implement the standardised approach. This is inconsistent with the current CRR rules; as a result, the draft RTS exceed their legal mandate in an inappropriate manner. We have extremely strong reservations over this fact. Unless this benchmark is abandoned in light of the above caveats, we assume that the supervisor will either perform the RWA calculation using the standardised approaches or that the supervisor will outsource the latter to a provider of reporting software.
(iii) comparison between estimates and outturns
First of all, EBA has to ensure that all participating IRBA-institutes use the same definitions of default. Furthermore it is obvious that a modification of the consideration of collaterals would lead to a change in Loss Given Default. Failure to deal with those two issues would lead to distorted results.
Calculating the RWA on historical defaults and losses involves two problems:
• Low default rates in wholesale portfolios lead to the problem that quite a lot rating classes do not contain a default (depending on the size of portfolio).
This is especially problematic for the better ratings classes. Consequently an aggregation of rating classes is necessary to receive reasonable results.
• To ensure a homogeneous determination of loss rates, there is a need for an EBA standard. Actually there is no specification given. Especially in the case of the one year horizon a determination of a realised loss is quite impossible because the duration of realisation is longer than the given horizon.
In our view, the use of "outturns" versus "estimates" gives rise to a host of methodological problems. For instance, already the number of the rating classes used in the IRBA might change the confidence intervals - and thus potentially a bank's conspicuousness. This may lead to wrong conclusions."
Q4. What in your view is the most appropriate benchmark and/or approach for the assessment of the level of potential underestimation of own funds requirements?
We suggest country specific peer group benchmarks.For the IRBA, it may be helpful to use estimates offered by pool data operators as a benchmark. In Germany, there are several such providers.
Q5. Which set of market risk portfolios do you consider more appropriate for the initial exercise conducted under Article 78?
The new EBA benchmark portfolio (annex VII a) seems to contain basic instruments to a larger extent than the TBG benchmark portfolio (Annex VII b), thus better representing the main drivers of RWAs in the banks’ real portfolios.Furthermore, we would welcome a limitation to plain vanilla instruments. This enhances comparability given that not all banks are able (and, what is more, do not have to be able) to perform a valuation of e.g. double-no-touch options and variance swaps. However, according to Annex VII a, this exercise appears to be clearly more comprehensive than the comparative computation by the Trading Book Group. There is a need to calculate more instruments in more portfolios featuring a greater number of combinations which clearly drives up the operational costs (instrument setup, portfolio setup and portfolio statement, the first two of which are partly purely manual). As far as cost-benefit aspects are concerned, less is clearly more at this point (in the absence of any sacrifice in terms of added insights).
One major advantage in our view is the option of an analysis at the level of risk factors. Prior to the actual exercise, the test of the portfolios should be performed in close cooperation with the banks affected. This way, potential inconsistencies and misunderstandings can be addressed in advance. Also, a Q&A process should be established.
Q6. As explained in the background section, do you consider the approach proposed by the EBA appropriate for future annual exercises?
In our view, the benchmarking exercise's limited meaningfulness as well as the very limited added insights it offers fail to justify the proposed deployment of major resources by banks and supervisors. This equally applies to the IT resources required for the implementation of the annual reporting process.Q7. Do you have any alternative proposals? If yes, please provide details.
First and foremost, the goal should consist in ensuring that the assessment criteria do not de facto lead to a situation where all banks are affected for all observed portfolios by the assessment. The EBA itself acknowledges that the fulfilment of the assessment criterion does not mean that the model estimates are faulty. Under such circumstances, however, in order to avoid major costs in the absence of any gain in added insight, the assessment criteria have to be selected particularly carefully. Hence we should like to issue the following recommendations:• The assessment criterion comparison to the standard approach" should be deleted (for a more detailed explanation cf. reply to Q3)
• The impact of the quartile-based assessment criterion should be mitigated by changed quantiles (e.g. 10%). Cf. also the answer to Q2 (box plots)."
Q8. Which of the two options for phasing-in do you consider preferable?
We prefer option 2 featuring portfolio rotation.Q9. Do you see any potential ambiguities in the credit risk portfolios defined in Annex I? Please identify the relevant portfolio providing details and any suggestions that would eliminate these ambiguities.
According to the definition of the clusters under Annex I/II C 103, the clusters for high default portfolios are inter alia formed on the basis of the so-called Indexed Loan To Value (ILTV). The ILTV is calculated on the basis of the current credit amount and the current market value. Yet, not all banks necessarily have access to the market value information. Under the provisions of the CRR, banks can either use the Market Value or the Loan to Value Ratio. On the basis of an EBA-RTS the latter is currently undergoing European wide standardisation.Please confirm that the counterparties are to be understood as the legal entities. I.e. if ACME Inc. is listed as counterparty in the sample portfolio, the information requested refers to the legal entity ACME Inc. and not to any other entities / subsidiaries in the group of connected clients to which ACME Inc. belongs.
With regard to the report on the consolidated basis, clarification should be provided on how to report credit risk parameters which are not identical within the banking group. E.g., the same counterparty may have different ratings in different subsidiaries of the banking group or there may be different approaches to quantifying collateral haircuts in different subsidiaries. One possible approach would be to use the parameters of the group company with the largest exposure share for the sample portfolio. This might however result in a mismatch between the one specified PD and the overall reported RWA amount (calculated with the actual diverging PDs).
The portfolio of large corporates is defined as “annual turnover > EUR 200 mln”. Banks’ internal rating models for corporates are likely to have deviating turnover categories. I.e. one specific rating system may contain corporates with a turnover above as well as below EUR 200 mln. For data such as “default rate past 5 years” banks should be allowed to deliver this figure based on the turnover categories of their corporate rating systems, because it would be a laborious additional operating procedure to calculate a 5 year past default rate based on a portfolio definition that does not correspond to the banks’ internal corporate rating system categories.
Q10. Do you have any suggestions for additional credit risk portfolios? Please provide details.
N/AQ11. Do you see any potential ambiguities in the market risk portfolios defined in Annexes VII.a and VII.b? Please identify the relevant portfolio providing details and any suggestions that would eliminate these.
In Annex VII.a the descriptions of instruments 30 and 31 show discrepancies in their individual definitions in speaking about DKK/USD resp. SEK/USD forwards but in the explanation about long DKK, short EUR" resp. "long USD, short EUR". In our opinion it should read "short DKK, long USD" resp. "short SEK, long USD" in correspondence to instrument 25."Q12. Do you have any suggestions for additional market risk portfolios? Please provide details.
N/AQ13 Do you agree with the possibility of allowing firms to refrain from reporting portfolios if one of the conditions stated in Article 3 is met?
We agree, but it should be clarified whether F-IRB banks would need to complete the information for LGD (and EAD and maturity). While F-IRB banks use supervisory values for these parameters, the LGD still has a bank-estimated component in the case of physical collateral. For physical collateral (e.g. real estate), the collateral value that feeds into the LGD calculation is determined by the bank.Q14 Do you have any suggestion about additional exemptions from reporting? If yes, please provide details.
The aim of the benchmarking portfolio process is to gain insights on differences in banks’ RWA calculation. Thus, banks should only calculate risks for those products and risk categories for which they have regulatory model approval (and only these results should be used for the regulatory benchmarking). This also has the advantage that all risk models used for the benchmarking process have undergone the same stringent regulatory approval process thus guaranteeing the application of comparable standards. The results will then be less distorted by effects which are not in the scope of the benchmarking exercise.Hence, banks featuring internal approaches which lack supervisory authorisation should be explicitly excluded from the benchmarking exercise. Missing authorisations have a considerable impact on the degree of divergence (this has been demonstrated e.g. by the results of the SIG-TB). Hence, they would render benchmarking approaches obsolete. Individual banks still lacked a supervisory authorisation for internal approaches concerning the additional default risks and migration risks in the so-called IRC portfolios. These banks considerably overestimate the risks; the latter only became possible given the missing use for the purposes of calculating capital charges.
Individual clusters may be immaterial for individual banks. In such cases, further exemptions should be possible. To this end, the EBA could stipulate a relative materiality threshold (e.g. 5% max. of the total RWA).