Response to consultation Paper on draft RTS and ITS on benchmarking portfolios
Go back
For market risk, benchmarking should only be done for portfolios where the bank actually trades in, or has trades existing. For some portfolios, one could consider to assess maximum loss figures (e.g. for long options one can in principle not lose more than the premium).
Regarding the proportionate aspect of the proposal, the competent authorities are expected to investigate any “output modelling values and standard deviation of the output modelling values falling in the first and fourth quartile of the peer’s ample distribution”. This rule would systematically require competent authorities to enquire on half of the contributions, it appears disproportionate.
On the comparison between estimates and outturns:
The comparison of estimates with outturns should provide a basis for supervisory engagement provided that the tests used are meaningful and, in particular, meet the challenge posed by rating philosophies. However, the comparison between estimates and outturns is very dependent on the number of rating classes. In fact, depending on the number of rating classes in use the confidence intervals, thus the RWA, could be quite different. In any case, EBA should give precise guidance on the computation of outturns (such as a spreadsheet), and these computations should take into account all components of the statistical validity of risk estimates.
On the first and fourth quartiles:
The combination of a first and fourth quartile approach could be useful if it is used only to drive initial supervisory interest as distinct from driving conclusions directly. If used for screening for a more in-depth assessment of the internal approaches the first and fourth quartiles would appear to be too broad a sample as they comprise 50 % of all banks. For that purpose it would be more appropriate to use a narrower sample, e.g. the first and last tenths (deciles).
On the comparison between own funds under internal models and standardised approach:
We see no meaningful role for benchmarks based upon standardised risk weights.
Article 78 does not mandate the EBA to define a universal absolute benchmark but rather to help competent authorities identifying those approaches “where there is a significant and systematic under- estimation of own funds requirements”. We consider this objective cannot be achieved on the basis of a simplistic indicator but rather by thorough analysis of model assumptions and performance. The necessity to define a common absolute benchmark is not obvious to achieve the benchmarking objectives, and the choice of the standard approach as an absolute benchmark is not relevant. An additional issue is that in some jurisdictions supervisors have implemented local regulatory discretions (such as the decision of the Belgian National Bank to increase the risk weight of Belgian mortgages), which may result in poor comparability of some portfolios.
For market risk, the benchmark should not be based on the standardised model, which is being reviewed anyway. The EBF advises the EBA to perform the benchmark based on the values of risk factors instead. We note that the EBA proposes that portfolios with specific relevance for EU non-Euro banks should be included in the market risk benchmarking. That is of course of interest, however it should be noted that there are only four banks in Denmark and Sweden altogether with IMAs and currently only one with specific risk approval. Therefore the outcome of the benchmarking will be very limited.
As regards the confidence level set at 97.5%, it could prove inappropriate on some portfolios (owing to various sample sizes, or to the weight of expert judgment in the rating process). We wonder whether there is evidence grounded on economic or academic studies establishing the relevance of such a confidence level.
appetites of individual firms. For market risk, the most appropriate tests are the internal back testing results. Benchmarking can add a lot of value in this domain, provided the differences in risk factors are well understood.
Analysts have published studies based on pillar 3 reports with comparisons between model forecasts and actual losses. Reportedly, most of the banks show actual losses way below the estimates across all portfolios even during the years after the wake of the crisis.
Evidence clearly indicates that the problem of modelling differences is not the underestimation of capital requirements but the difficulties posed in the comparative analysis of institutions. This needs to be clearly understood by analysts, investors, the media and other stakeholders.
However, for banks which do not already participate in the exercise of the Basel Committee the second option could be unduly burdensome as it would entail portfolio set up costs in order first to apply the Basel portfolio for the 2014 exercise and later new portfolio set up costs in order to apply an EBA portfolio for 2015 onwards. We therefore urge the EBA to consider the possibility of exempting banks which do not already participate in the exercise of the Basel Committee from the 2014 exercise or, alternatively, to develop the EBA portfolio to be used from the outset, while allowing banks already participating in the Basel exercise the option to use the Basel portfolio for the 2014 exercise.
Due to the current divergent background of practices the current benchmarking proposal for credit risk hardly justifies its cost with the limited utility it has to offer. In order to achieve a more fruitful and meaningful step towards consistency of internal approaches, additional guidance on these approaches should be developed. The role of benchmarking can then develop in a more meaningful way.
As to the design of the benchmark portfolio, it seems to EBF members that the use of non-existing transactions is unrealistic and should therefore be reconsidered. Banks should use transaction types present in their portfolios otherwise the meaningfulness of the benchmark exercise is questionable. Risk assessment is not a mere calculation engine but it is a wider discipline involving qualitative assessment, internal controls, governance and other features; the quantitative element is just one more part of risk management. It is important to analyse and understand the root causes of the differences.
We would like to focus our comments on the use of HPE for LDP: this approach seems very hard to use, especially as far as secured LGD is concerned, mainly for these reasons:
In contrast to previous HPE which used existing transactions, this one proposes underlying transactions that do not exist in the bank’s portfolio. This would put into question the reliability of estimates and would materially increase the workload.
It is highly unlikely that banks can provide all the requested information to determine CCF/LGD Secured. LGD secured is not function of the collateral itself but it also depends on other factors including:
o the type of collateral, location, and condition;
o its accessibility (e.g. seniority of the claim, legal environment, nature of the counterparty);
o the characteristics of the loan (e.g. type, LTV);
o the purpose of the asset.
The outputs would depend on the zone of expertise of some institutions: for instance, a bank would be able to give an LGD estimate for a real estate transaction in its country, whereas it would probably be much more difficult for a foreign retail bank.
The value of the approach is felt as limited, mainly owing to representativeness issues. We also would like to draw attention to the fact that combining this approach with others for benchmarking purposes would become really burdensome for banks with LDP.
For market risk, the suggested approach appears to be logical and having an assessment of the quality of market risk models is supported. As the Basel Committee has already concluded that the current market risk framework should be revised, the information gathered from benchmarking exercises such as this one would be beneficial for the quality of the fundamental review of the trading book.
In any case, closer coordination on timing and content of HPE requirements and exercises would be useful. Also, submission of results should not take place until firms have come to a consensus on position specification.
The less costly;
The one with the longest lead-in time.
For market risk, the actual portfolios should be used. If it is decided to use new ones instead, there should be good test procedures in order to ensure high portfolio quality.
In the case of low-default-portfolios (LDP) the classes of assets included is somewhat ambiguous. For instance, there is no clarity as to whether and how specialised lending should be included in the LDP benchmarking.
Under appendix 1, tab 103 requires data for retail and SME mortgage assets. Column D specifies the “residence of counterparties”. The drop down options of the “Residence of counterparties” are only European Countries. Is it correct to assume that non-European Country related data is exempted from the benchmarking?
General comments regarding benchmark portfolios:
Portfolios should to be available well in advance of the exercises. They should be extensively tested by the competent authority to avoid any confusion or late changes to the portfolios. If possible, a rough indication of the MtM should be available at the moment of the publication of the portfolios.
The competent authorities should structure the portfolios in such a way that the specific model item can be properly tested. For instance, if the purpose is to test VaR models for interest rate volatility, the portfolio should be structured in such a way that all non-vega sensitivities of the portfolio are small. In this specific case for example the portfolio with a swaption can be made delta neutral by adding an interest rate swap.
As regards ambiguities, the definition of portfolios in terms of how to combine the various positions (section 3, Annex Vii.a) should be clarified. For example, portfolio II is described as 1-50 instruments, 2-9 instruments and 3-1 instrument. We understood 1-50 instruments to mean long all positions labelled as 1-50 in section 2.
A spreadsheet list of which instruments to include in each sub-portfolio would be clearer as well as including the possibilities of being long or short the position described in section 2 and in multiples of the quantity described. For example:
Instruments
1. Long Index X OTC Future (1 point equals 10 € movement). Expiry - June 2014
2. Long OTC Future Bank A (1 contract = 100 shares). Expiry – June 2014
3. Long OTC Future Bank B (1 contract = 100 shares). Expiry – June 2014
4. Long OTC future, Bank C (1 contract = 100 shares). Expiry – 30 June 2014
5. Long OTC future, Bank D (1 contract = 100 shares). Expiry – 30 June 2014
6. Long OTC future, Bank E (1 contract = 100 shares). Expiry – 30 June 2014
7. Long OTC call Option. Underlying Corporate 1, ATM (1 contract = 100 shares). Expiry – 31 July 2014
8. Long OTC call Option. Underlying Corporate 1, ATM (1 contract = 100 shares). Expiry – 31 December 2014
9. Long OTC Future Index Y CAP. Expiry – 30 June 2014
10. Long call Option. Underlying Corporate 2, ATM (1 contract = 100 shares). Expiry – 31 July 2014
Portfolios
I Long 1-10.
II Long 1-5 and short 6-10.
III Long 1-3 and short 8-10.
Definition of positions using some kind of trade blotter would help eliminate position definition operational errors as it would encourage copying and pasting or other electronic form of trade entry rather than keying in the data.
While we reiterate our preference to use the Basel portfolios as outlined in the response to Q5. However, in case option two would also be used, we would propose introduction of an FX vanilla option out of the money with strike far from the FX forward by 2.33 standard-deviations for FX test portfolios. The goal being to check if the VaR captures the convexity between the current FX Spot and the 99% FX Spot bump. For illustration, the related instrument could be:
Sell call EUR put USD with strike = Current FX Fwd x (1 + 1%) and sell put EUR call USD with strike = Current FX Fwd x (1 - 1%).
Another case that could grant an exemption is when an institution is in the process of purchasing another financial institution until decisions are made and authorised regarding the merger of portfolios.
It would be welcomed to envisage also an exemption for credit risk. As for market risk, the exemptions should be based on situations where the institutions are under a model validation process for the portfolios included in the samples or for non-material portfolios. As such, a materiality threshold could be defined as:
An absolute portfolio size;
A relative portfolio size, in comparison to the total consolidated balance-sheet or to the balance-sheet size of the subsidiary.
Portfolios with partial roll-out should also be exempted (or alternatively, the share of the portfolio under Standard approach should be highlighted and the bias resulting from different capital treatments should be eliminated).
Local entities supervised by a host supervisor should also be exempted from solo reporting as long as their portfolios are included in the consolidated vision submitted to the home supervisor.
Q2. Do you consider that the benchmarks outlined in the RTS are sufficiently proportionate and flexible? Do you have any alternative benchmark proposals? If yes, please provide details.
The classes of benchmarks outlined in the RTS appear sufficiently flexible. Following our line of thought that benchmarking should be preceded by the adoption of common approaches to model calibration, the collection of benchmark data should be kept as minimal as possible until such commonality in model calibration and defaulted assets is established.For market risk, benchmarking should only be done for portfolios where the bank actually trades in, or has trades existing. For some portfolios, one could consider to assess maximum loss figures (e.g. for long options one can in principle not lose more than the premium).
Regarding the proportionate aspect of the proposal, the competent authorities are expected to investigate any “output modelling values and standard deviation of the output modelling values falling in the first and fourth quartile of the peer’s ample distribution”. This rule would systematically require competent authorities to enquire on half of the contributions, it appears disproportionate.
Q3. What limitations do you see in relation to the use of the proposed benchmarks, i.e., (i) first and the fourth quartiles; (ii) comparison between own funds under the internal models and the standardised approach; and (iii) comparison between estimates and outturns?
Given the current dispersion of supervisory practices the utility of the benchmarks proposed is quite limited. Once further commonality and understanding is available we think the following:On the comparison between estimates and outturns:
The comparison of estimates with outturns should provide a basis for supervisory engagement provided that the tests used are meaningful and, in particular, meet the challenge posed by rating philosophies. However, the comparison between estimates and outturns is very dependent on the number of rating classes. In fact, depending on the number of rating classes in use the confidence intervals, thus the RWA, could be quite different. In any case, EBA should give precise guidance on the computation of outturns (such as a spreadsheet), and these computations should take into account all components of the statistical validity of risk estimates.
On the first and fourth quartiles:
The combination of a first and fourth quartile approach could be useful if it is used only to drive initial supervisory interest as distinct from driving conclusions directly. If used for screening for a more in-depth assessment of the internal approaches the first and fourth quartiles would appear to be too broad a sample as they comprise 50 % of all banks. For that purpose it would be more appropriate to use a narrower sample, e.g. the first and last tenths (deciles).
On the comparison between own funds under internal models and standardised approach:
We see no meaningful role for benchmarks based upon standardised risk weights.
Article 78 does not mandate the EBA to define a universal absolute benchmark but rather to help competent authorities identifying those approaches “where there is a significant and systematic under- estimation of own funds requirements”. We consider this objective cannot be achieved on the basis of a simplistic indicator but rather by thorough analysis of model assumptions and performance. The necessity to define a common absolute benchmark is not obvious to achieve the benchmarking objectives, and the choice of the standard approach as an absolute benchmark is not relevant. An additional issue is that in some jurisdictions supervisors have implemented local regulatory discretions (such as the decision of the Belgian National Bank to increase the risk weight of Belgian mortgages), which may result in poor comparability of some portfolios.
For market risk, the benchmark should not be based on the standardised model, which is being reviewed anyway. The EBF advises the EBA to perform the benchmark based on the values of risk factors instead. We note that the EBA proposes that portfolios with specific relevance for EU non-Euro banks should be included in the market risk benchmarking. That is of course of interest, however it should be noted that there are only four banks in Denmark and Sweden altogether with IMAs and currently only one with specific risk approval. Therefore the outcome of the benchmarking will be very limited.
As regards the confidence level set at 97.5%, it could prove inappropriate on some portfolios (owing to various sample sizes, or to the weight of expert judgment in the rating process). We wonder whether there is evidence grounded on economic or academic studies establishing the relevance of such a confidence level.
Q4. What in your view is the most appropriate benchmark and/or approach for the assessment of the level of potential underestimation of own funds requirements?
For credit risk models, the most appropriate approach to test potential underestimation of own funds requirements is the comparison between models’ estimates of risk parameters (i.e. PD, LGD and CF) and actual outturns in these parameters. As discussed above, such tests should account for the philosophy of the risk parameter (e.g. through-the-cycle) and the current economic environment. This approach has the advantage of naturally reflecting the riskappetites of individual firms. For market risk, the most appropriate tests are the internal back testing results. Benchmarking can add a lot of value in this domain, provided the differences in risk factors are well understood.
Analysts have published studies based on pillar 3 reports with comparisons between model forecasts and actual losses. Reportedly, most of the banks show actual losses way below the estimates across all portfolios even during the years after the wake of the crisis.
Evidence clearly indicates that the problem of modelling differences is not the underestimation of capital requirements but the difficulties posed in the comparative analysis of institutions. This needs to be clearly understood by analysts, investors, the media and other stakeholders.
Q5. Which set of market risk portfolios do you consider more appropriate for the initial exercise conducted under Article 78?
Regarding the options presented in the consultative document for market risk internal models; feedback received from our members clearly indicates that the EBF would favour the second option, namely the use of the same portfolios as in the hypothetical portfolio exercises of the Basel Committee and the EBA. Based on the experience of our members, using existing high quality portfolios will produce better results and reduce the development effort required.However, for banks which do not already participate in the exercise of the Basel Committee the second option could be unduly burdensome as it would entail portfolio set up costs in order first to apply the Basel portfolio for the 2014 exercise and later new portfolio set up costs in order to apply an EBA portfolio for 2015 onwards. We therefore urge the EBA to consider the possibility of exempting banks which do not already participate in the exercise of the Basel Committee from the 2014 exercise or, alternatively, to develop the EBA portfolio to be used from the outset, while allowing banks already participating in the Basel exercise the option to use the Basel portfolio for the 2014 exercise.
Q6. As explained in the background section, do you consider the approach proposed by the EBA appropriate for future annual exercises?
Firstly, we would like to request the EBA to deliver portfolio specifications to banks well in advance of the exercises. Booking positions in test portfolios, checking them and performing validation processes requires a minimum time for preparatory tasks of at least 6 months. It would also be important to avoid multiple iterations of instructions or later changes to the portfolios.Due to the current divergent background of practices the current benchmarking proposal for credit risk hardly justifies its cost with the limited utility it has to offer. In order to achieve a more fruitful and meaningful step towards consistency of internal approaches, additional guidance on these approaches should be developed. The role of benchmarking can then develop in a more meaningful way.
As to the design of the benchmark portfolio, it seems to EBF members that the use of non-existing transactions is unrealistic and should therefore be reconsidered. Banks should use transaction types present in their portfolios otherwise the meaningfulness of the benchmark exercise is questionable. Risk assessment is not a mere calculation engine but it is a wider discipline involving qualitative assessment, internal controls, governance and other features; the quantitative element is just one more part of risk management. It is important to analyse and understand the root causes of the differences.
We would like to focus our comments on the use of HPE for LDP: this approach seems very hard to use, especially as far as secured LGD is concerned, mainly for these reasons:
In contrast to previous HPE which used existing transactions, this one proposes underlying transactions that do not exist in the bank’s portfolio. This would put into question the reliability of estimates and would materially increase the workload.
It is highly unlikely that banks can provide all the requested information to determine CCF/LGD Secured. LGD secured is not function of the collateral itself but it also depends on other factors including:
o the type of collateral, location, and condition;
o its accessibility (e.g. seniority of the claim, legal environment, nature of the counterparty);
o the characteristics of the loan (e.g. type, LTV);
o the purpose of the asset.
The outputs would depend on the zone of expertise of some institutions: for instance, a bank would be able to give an LGD estimate for a real estate transaction in its country, whereas it would probably be much more difficult for a foreign retail bank.
The value of the approach is felt as limited, mainly owing to representativeness issues. We also would like to draw attention to the fact that combining this approach with others for benchmarking purposes would become really burdensome for banks with LDP.
Q7. Do you have any alternative proposals? If yes, please provide details.
For credit risk, an alternative trigger could be the use of country-specific industry means or medians together with a definition of an absolute or relative acceptable variation from the mean or median (to be defined by EBA or national competent authorities). This approach appears particularly relevant for the benchmarking exercises on mortgage portfolios. The divergences of RWA on this portfolio are mainly explained by the legal specificities (i.e. state guarantee) and supervisory discretions.For market risk, the suggested approach appears to be logical and having an assessment of the quality of market risk models is supported. As the Basel Committee has already concluded that the current market risk framework should be revised, the information gathered from benchmarking exercises such as this one would be beneficial for the quality of the fundamental review of the trading book.
In any case, closer coordination on timing and content of HPE requirements and exercises would be useful. Also, submission of results should not take place until firms have come to a consensus on position specification.
Q8. Which of the two options for phasing-in do you consider preferable?
The vast majority of EBF members think that the phase-in option (number 2) is preferable on the grounds that it is: The less costly;
The one with the longest lead-in time.
For market risk, the actual portfolios should be used. If it is decided to use new ones instead, there should be good test procedures in order to ensure high portfolio quality.
Q9. Do you see any potential ambiguities in the credit risk portfolios defined in Annex I? Please identify the relevant portfolio providing details and any suggestions that would eliminate these ambiguities.
Regarding residential mortgages, it would be worth considering specificities that can materially change the risk profile like the different forms of state guarantee programs. Evidence found in the EBA fourth analysis on the consistency of risk weighted assets clearly indicates that member states in which a state guarantee is commonplace have different risk drivers. The credit losses experience is also different and can be explained with long-term observed data. Therefore, it would make sense to split the portfolio and analyse separately with bespoke benchmarks the transactions that are backed with such a state guarantee.In the case of low-default-portfolios (LDP) the classes of assets included is somewhat ambiguous. For instance, there is no clarity as to whether and how specialised lending should be included in the LDP benchmarking.
Under appendix 1, tab 103 requires data for retail and SME mortgage assets. Column D specifies the “residence of counterparties”. The drop down options of the “Residence of counterparties” are only European Countries. Is it correct to assume that non-European Country related data is exempted from the benchmarking?
Q10. Do you have any suggestions for additional credit risk portfolios? Please provide details.
We do not have further suggestions for additional credit risk portfolios at this time.Q11. Do you see any potential ambiguities in the market risk portfolios defined in Annexes VII.a and VII.b? Please identify the relevant portfolio providing details and any suggestions that would eliminate these.
Market risk proposals indicate that banks should calculate risk figures in the currency of the particular portfolio specified in order for the result not to be polluted by currency effects. However, models of non-Euro markets (e.g. Denmark or Sweden) are in some cases set up to calculate with an outset in Euro therefore the conversion is a time-consuming complication. In our view, it should not be generally expected that banks be able to do this.General comments regarding benchmark portfolios:
Portfolios should to be available well in advance of the exercises. They should be extensively tested by the competent authority to avoid any confusion or late changes to the portfolios. If possible, a rough indication of the MtM should be available at the moment of the publication of the portfolios.
The competent authorities should structure the portfolios in such a way that the specific model item can be properly tested. For instance, if the purpose is to test VaR models for interest rate volatility, the portfolio should be structured in such a way that all non-vega sensitivities of the portfolio are small. In this specific case for example the portfolio with a swaption can be made delta neutral by adding an interest rate swap.
As regards ambiguities, the definition of portfolios in terms of how to combine the various positions (section 3, Annex Vii.a) should be clarified. For example, portfolio II is described as 1-50 instruments, 2-9 instruments and 3-1 instrument. We understood 1-50 instruments to mean long all positions labelled as 1-50 in section 2.
A spreadsheet list of which instruments to include in each sub-portfolio would be clearer as well as including the possibilities of being long or short the position described in section 2 and in multiples of the quantity described. For example:
Instruments
1. Long Index X OTC Future (1 point equals 10 € movement). Expiry - June 2014
2. Long OTC Future Bank A (1 contract = 100 shares). Expiry – June 2014
3. Long OTC Future Bank B (1 contract = 100 shares). Expiry – June 2014
4. Long OTC future, Bank C (1 contract = 100 shares). Expiry – 30 June 2014
5. Long OTC future, Bank D (1 contract = 100 shares). Expiry – 30 June 2014
6. Long OTC future, Bank E (1 contract = 100 shares). Expiry – 30 June 2014
7. Long OTC call Option. Underlying Corporate 1, ATM (1 contract = 100 shares). Expiry – 31 July 2014
8. Long OTC call Option. Underlying Corporate 1, ATM (1 contract = 100 shares). Expiry – 31 December 2014
9. Long OTC Future Index Y CAP. Expiry – 30 June 2014
10. Long call Option. Underlying Corporate 2, ATM (1 contract = 100 shares). Expiry – 31 July 2014
Portfolios
I Long 1-10.
II Long 1-5 and short 6-10.
III Long 1-3 and short 8-10.
Definition of positions using some kind of trade blotter would help eliminate position definition operational errors as it would encourage copying and pasting or other electronic form of trade entry rather than keying in the data.
Q12. Do you have any suggestions for additional market risk portfolios? Please provide details.
As the risks for most banks are plain vanilla, the EBF suggests replacing exotic portfolios by vanilla portfolios.While we reiterate our preference to use the Basel portfolios as outlined in the response to Q5. However, in case option two would also be used, we would propose introduction of an FX vanilla option out of the money with strike far from the FX forward by 2.33 standard-deviations for FX test portfolios. The goal being to check if the VaR captures the convexity between the current FX Spot and the 99% FX Spot bump. For illustration, the related instrument could be:
Sell call EUR put USD with strike = Current FX Fwd x (1 + 1%) and sell put EUR call USD with strike = Current FX Fwd x (1 - 1%).
Q13 Do you agree with the possibility of allowing firms to refrain from reporting portfolios if one of the conditions stated in Article 3 is met?
Yes, banks should be exempted from reporting requirements as outlined in Article 3 if it is well justified and the criteria are transparent.Q14 Do you have any suggestion about additional exemptions from reporting? If yes, please provide details.
In addition, it should be noted that individual clusters as defined may be immaterial for certain banks. In such cases, banks should also be afforded exemptions from reporting where to do so is overly burdensome.Another case that could grant an exemption is when an institution is in the process of purchasing another financial institution until decisions are made and authorised regarding the merger of portfolios.
It would be welcomed to envisage also an exemption for credit risk. As for market risk, the exemptions should be based on situations where the institutions are under a model validation process for the portfolios included in the samples or for non-material portfolios. As such, a materiality threshold could be defined as:
An absolute portfolio size;
A relative portfolio size, in comparison to the total consolidated balance-sheet or to the balance-sheet size of the subsidiary.
Portfolios with partial roll-out should also be exempted (or alternatively, the share of the portfolio under Standard approach should be highlighted and the bias resulting from different capital treatments should be eliminated).
Local entities supervised by a host supervisor should also be exempted from solo reporting as long as their portfolios are included in the consolidated vision submitted to the home supervisor.