Response to consultation Paper on draft RTS and ITS on benchmarking portfolios
Go back
To the extent that NCAs are expected to investigate any “outlier” and provide the EBA with feedback, we believe that the proposed benchmarks are overly prescriptive and disproportionate. For the sake of illustration, the “1st and 4th quartile” rule would systematically require competent authorities to investigate half of the contributions. At least a more selective distribution of fractiles (e.g. 5% lowest and 5% highest values), or asymmetric fractiles (e.g. 10% lowest, and 2% highest) could be contemplated as triggers for abnormal calibrations. Similarly, the “80% SA output criteria” is likely to result in most contributions being considered as outliers for market risk test portfolios with diversification/netting features.
Moreover, as the EBA is going to generate the results of the benchmarking based on NCA conclusions (Article 2, RTS), the criteria used by the different NCAs must be comparable. This comparability must be assured by the EBA through a more detailed specification (i.e. more detailed than in the current RTS).
It is also worth mentioning that the benefits of diversification (portfolio and geographical diversification) should be considered within the benchmarking assessment.
Alternative proposals are discussed in answers to questions 3 and 4 below.
We recognize the importance of using common and objective benchmarks to identify extreme values. However, we consider the current criterion (1st and 4th quartile irrespective of the dispersion and clustering of the contributions) unsuitable. This is because it can lead to the identification of a contributor being “mechanically” an outlier while, by any reasonable materiality standard, it would not be considered so. For example, in cases of low dispersion where all the contributed results are very close to each other (say, within 1% of each other), 50% of the contributors based on the 1st and 4th quartile rule will still be identified as outliers. Take another example where all contributors have results that are very close together, except for one that is much lower than the others. The 1st and 4th quartile rule would correctly include this latter contributor as an outlier, but it will also include contributors on the high side even though their results might just be marginally higher than most of the firms. To summarise, there is in our view no need to identify outliers when the dispersion is tight and there is no reason why the number of outliers on both side of the distribution should be equal.
Not only would the identification of many these types of false outliers be burdensome for both the regulators and the firms, the 1st and 4th quartile rule can also potentially lead to instability as whoever is identified as a outlier on the downside would want to use a model that generates more conservative results to exit that status. By doing so, next time around, it will not appear as an outlier, while another firm who was previously just within the quartile cutoff might then be pushed out even though it will not have done anything differently.
To overcome these issues, we propose to thin out the percentile range for indentifying potential outliers on top of which we would overlay a materiality rule. Specifically, as a first step, we would identify the top 10% and bottom 10% as potential outliers. Secondly, we would then check if their results are more than x% away (say 20%) from the median. Only those contributors with results more than x% away from the median would ultimately be retained as outliers.
The advantage of this modified rule is that it takes into account both relative ordering and absolute differences. If the dispersion is very tight and no contributor has results more than x% away from the median, than there will be no outlier. However, if there is only one outlier far away from everyone else, it will have been correctly identified without any other contributors inadvertently being called outliers when in fact they are not.
(ii) Comparison between own funds under the internal models and the standardized approach
As explained in our answer to question Q1, our view is that it is not appropriate to define a common reference level of own funds at test portfolio level. Moreover, comparing required own funds calculated under an internal model and the SA could be misleading due to inherent simplifications, omissions and approximations in the parameterization of the SA where sound risk management practices (including business strategies, recovery policies, the presence of collateral, the use of hedging etc.) are not taken into account. By way of example, within the course of the Phase 2 SIG TB market risk benchmarking exercise, CRM floor contributions were even more volatile than CRM contributions themselves.
In addition to being risk insensitive, SA metrics are not universal standards in their implementation across jurisdictions. Not only could the interpretation of the same standards differ across banks, a number of factors under this approach are also open to national supervisory discretion. For instance, the treatment of sovereign risk under the SA is subject to national discretion and a lower risk weight can be applied to banks’ exposures to the sovereign of incorporation denominated in domestic currency and funded in that currency.
We therefore believe that the appropriate level of capital for a given portfolio is firm-specific and must be assessed in the light of the firm’s ability to risk-manage such a portfolio, the performance of the internal model and the representativeness of the portfolio with respect to the overall internal model.As far as market risk is concerned, should the EBA ultimately choose to maintain SA metrics as a common benchmark, we urge the EBA to consider at a minimum the following:
o The application of SA output as a benchmark should be postponed until the new SA under the Fundamental Review of the Trading Book (FRTB) is implemented for market risk,
o The computation of SA metrics should be performed at EBA level and the methodology used made available to all participating banks. This would be particularly useful for certain specific portfolios where national implementation of SA metrics diverges.
(iii) Comparison between estimates and outturns
Before comparing estimates and outturns, one must assess whether they refer to comparable economic situations. We consider that it is important to compare regulatory parameters (point-in-time, through-the-cycle or downturn) with outturns from the current situation to assess if banks from the same country are equally affected by the business cycle. Such a comparison is a way to assess differences in the dynamics of parameter estimates.
Furthermore, the EBA should give precise technical guidance on the computation of outturns (for instance in the form of a spreadsheet), and these computations should take into account all components of the statistical validity of risk estimates, such as the number of years of available history, default sample size, number of models in use for the respective cluster, etc. This guidance should be developed in consultation with the industry to ensure its suitability and that it can be replicated for institutions across Member States.
Consider for example a portfolio where origination has decreased over past years due to prudent risk management. Perhaps counterintuitively, the cost of risk on the portfolio is likely to have risen over a 5-year history (as the number of contracts in the portfolio will have decreased). However, the relative information within the 5 individual, annual default rates is not identical due to the change in risk profile and risk appetite. We would welcome guidance on how should such changes should be reflected in confidence intervals.
Additionally, with respect to credit risk, the use of the 97.5% confidence level is, in our opinion, a new paradigm which may prove inappropriate on some portfolios (for instance owing to various sample sizes or the weight of expert judgment in the rating process). We are not aware of economic or academic studies proving the importance or relevance of such a confidence level.
Finally we believe that competent authorities should not consider the different criteria as single “show-stoppers” but instead they should base their analysis on the combination of their outcome.
We consider that the only truly robust and valuable approach to identifying the potential under-estimation of own funds requirements is one based on the comparison of model-estimated values and realized values, i.e. backtesting. Nevertheless, NCAs should also use backtesting in conjunction with other sources of investigation like modeling construction and analysis as well as validation history (local supervisory practices and decisions on particular models). Lastly, we wish to stress that benchmarking can never be a substitute for NCAs exercising expert judgment.
Market risk
In order to help identify RWA variance drivers, our proposal is to subject market risk test portfolios to backtesting consistently with the Basel FRTB approach (which introduces backtesting at a much more granular level).
Consistently with its mandate, the EBA’s work should be annual and aligned with the timeline of the Basel monitoring exercises.
Credit risk
Lending practices are firm specific and model output and losses will also therefore be firm specific. The application of different accounting frameworks and a firms’ accounting policies will also mean that the measurement of losses for the same exposure may be different. It is important that the assessment of firms’ responses factors in these differences.
Along the same lines, in some cases NCAs have introduced floors or restrictions on the parameters that firms may use in internal model approaches. Some of these floors may be applied across all firms within a Member State, whilst other floors may be applied on a firm-by-firm basis. We would recommend that the EBA controls for such differences otherwise they may incorrectly be attributed to firms’ models and skew the results of the exercise.
Additionally, to avoid creating false outliers, expected losses (%) or RW (%) rather than PDs/LGDs should be the first point of comparison. Otherwise, two banks, one with a very early default definition and the other with a very late one, will tend to have high PD/low LGD and (respectively) low PD/high LGD whereas the results in EL% or RW% could be very similar.
We understand the assessment of internal models is one of the objectives of the “General provisions” laid out in Article 8 of the RTS. Nevertheless we do not fully understand to what extent the NCAs will make use of the information requested in Article 8 paras 1 and 2 of the RTS and would welcome clarification that the EBA will use this information to validate firm specific outcomes.
To conclude this section, we wish to stress that benchmarking exercises of banks’ risk assessment must be performed taking model performance into account and by backtesting outputs. It must be well understood that being an outlier in the context of the RWA benchmarking exercises does not necessarily imply that models are not performing appropriately. In fact, a well-performing model means that a bank is doing a good job in predicting its risks and assessing its capital, but it can still produce outlier results in RWA benchmarking exercises.
Given other ongoing regulatory exercises with overlapping timelines that many market risk experts have been focusing on (e.g. FRTB QIS, BCBS/EBA joint counterparty risk and CVA benchmarking exercise to mention only two such examples) and the very tight timeframe and close kick-off date contemplated by the EBA for the 2014 exercise, we consider that the conditions are not met to ensure the 2014 exercise can be properly performed on the basis of a new set of portfolios
Given the tight time frame referred to above, we favor the use of HPE portfolios for the first exercise of Q4 2014. Starting from the 2015 exercise, we could then switch to the new portfolios proposed by the EBA.
Moreover, we fully support the EBA’s proposition to organize a pre-validation phase to ensure correct booking of instruments. Before the start of any exercise, NCAs should ensure that initial market valuations are within an acceptable range and questions concerning instructions are resolved. Previous exercises such as the FRTB QIS on HPE showed that this pre-validation phase is of key importance to limit the risk of operational errors.
Lastly, we want to draw the EBA’s attention on the necessity for banks to receive portfolios specifications well in advance of the exercise. Booking positions in test portfolios, checking them and performing process validation requires time which cannot be reduced. Multiple iterations of instructions should be avoided to the extent possible to limit confusion or late changes to the portfolios.
Credit Risk
We note that previous public RWA benchmarking exercises have in particular highlighted the impact of banks' risk profiles and defaulted assets as being amongst the key drivers behind RWA variability. When benchmarking institutions, the effect of defaulted assets on a bank’s global RWA should be controlled or neutralized because, all else being equal, two banks with two different amounts of defaulted assets in their balance sheets will show two diverging RWA.
The benchmarking analysis should also take into account the type of banks being compared: a universal bank and a pure CIB bank may have legitimate divergent assessments of the RWA of the same assets, due to their different specializations.
This being said, we support the approach suggested by the EBA to alternate benchmarking exercises on LDP and HDP. This is because a thorough examination of the results, followed by explanations between supervisors and banks will obviously be a lengthy process, and (at least on the credit risk side) loss trends emerge progressively: macroeconomic changes are the main risk drivers, and will take time to materialize in loss parameters.
The HPE approach for LGD seems to be particularly burdensome to implement, especially as far as secured LGD is concerned, for at least three main reasons:
- This hypothetical exercise would be a true hypothetical exercise (compared to previous HPE exercises that have relied on existing information), since the underlying transactions would not exist. Therefore, the reliability of estimates would be questionable and the workload for banks much higher.
- It is unlikely that banks will be able to provide all the requested information to determine CCF/LGD Secured. LGD Secured is not just a simple function of the collateral only, it also depends on many different factors, including, obviously, the collateral (type, location, condition, etc.) but also the ability to have access to it (seniority of the claim, legal environment, nature of the counterparty etc.), the characteristic of the loan (type, LTV etc.), the usage of the asset, etc.
- The outputs would also depend on the area of expertise of some institutions: for instance, a German bank would be able to give an LGD estimate for a German real estate transaction, whereas it would probably be much more difficult for a retail Spanish bank to give an LGD for a German transaction.
The concept of cluster appears to be a good tool to tackle issues of representativeness and comparability of data collected across banks in the benchmarking process. Nevertheless, clusters should be defined pragmatically along drivers that are applicable to the majority of banks and with appropriate thresholds of materiality. Currently, we are concerned that the level of granularity set out in the EBA’s proposals will produce immaterial and unusable clusters.
As for HDP, we consider that the portfolios defined for mortgages are far too granular:
- Until banks set up/adapt an adequate IT solution for reporting for purposes of the benchmarking exercise, all of the requested information may not be directly available from all firms’ current IT systems (LTV buckets for instance might be recorded in another business or IT management environment).
- By selecting too many, excessively granular clusters, especially on retail exposures, comparability might be difficult or in vain, all the more so that these portfolios are more sensitive to local specificities.
- Very small clusters would not fit with the range of application of banks’ internal models: for instance one cluster could be covered by 2 models which makes backtesting difficult to implement and interpret.
Counterparty Risk
We reiterate that the consultation paper contains little information on the proposed counterparty risk portfolios that would be used for future annual exercises.
- The first step of any benchmarking exercise should be to establish what the key drivers behind RWA divergences are.
- Benchmarks should be expressed in terms of ranges of acceptable divergences rather than single, simplistic values.
- Benchmarking frameworks should be refined by segmentation according to the relevant key drivers of divergences such as, but not limited to, bank type, asset classes, geographical sector, etc.
Nevertheless, we would already like to provide the following comments/questions:
Annex I
- All: Col 060 - We assume that banks would be required to provide the PDs behind the rating scale to support the backtesting.
- All: Col 490 - We consider secured categories to include exposures that are partially secured to align with LGD modeling for single exposures.
- C101 - Portfolio Name & Sector do not seem to be always compatible: e.g. in the “Large Corporate Sample” portfolio, sectors like “Credit institutions” should not be allowed.
- C101: Col 450 - We consider country of residency to be country of operation.
- C102 / C103: Col 130 = Type of collateral is a mixture of collateral and personal guarantees. It is possible to have a personal guarantee or a credit derivative while having physical or financial collateral. Our proposal is to separate collateral and guarantees.
- C102 / C104: Col 270 = facilities like credit lines or credit cards may have both on balance and off balance exposures.
- C103: Col 020 - The templates are incomplete as they do not include Retail Other and Retail QRRE portfolios.
- C103: Col 160 - It is not appropriate to provide maturity for Retail Portfolios as it is not used to calculate capital requirements.
- C103: Col 450: - Annex I considers different countries inside the EU. Annex II mentions only EU/Non-EU options.
- C103: Col 460 - The definition of construction and the rationale for requesting a separate cluster for it and not for other lending types is not clear. For example, it would not capture all forms of lending secured on real estate.
- C104: Col 130 = Seems to be a mixture of real collateral, credit protection and seniority. In our opinion those are different dimensions and should be treated separately. Additionally, In our view negative pledges are not always enforced and / or included in LGD modeling so do not effect recoveries. This should be removed from the benchmarking exercises.
- C104: Col
Annex V
C105: Col 630 - requires firms to provide Long-run PD but this is not used by all institutions - for example Point-in Time PD models
Counterparty credit risk
We note that the EBA proposes to align with the Basel SIG TB exercise for the benchmarking of counterparty credit risk in 2014 to cover IMM and CVA models. We are concerned that the EBA paper contains little information on the proposed counterparty risk portfolios for 2014 or on what portfolios would be used from 2015.
It is also not clear what standards of assessment or reporting will be applied for counterparty risk and CVA as these are not detailed in the consultation paper and more specifically in neither the RTS or ITS. For example, for CVA, it is not clear whether the exercise will be required to capture both advanced and standardized calculations, or whether similar requirements for market risk are relevant, such as initial market valuations.
The EBA paper refers to the ‘scarcity of resources and the existing workload both for banks and NCAs. We would agree with EBA’s concern about resources and would propose that the 2014 exercise cover only the IMM component of counterparty risk and not CVA. Further, we would recommend that only one asset class be covered in the first year. Benchmarking interest rate products, for example, would cover a significant amount of the exposure and would allow for a meaningful comparison to be made between firms. This would be analogous to the phased approach for credit risk supported by the Associations (i.e. where EBA propose to perform the analysis of High Default Portfolios (HDP) and Low Default Portfolios (LDP) in alternate year).
We note that the counterparty risk sections in Annexes VIII and IX are incomplete. When the EBA is able to publish details on the counterparty risk benchmarking proposals, we would recommend that the EBA considers defining consistent margin terms and collateralisation agreements. Our members would be happy to help define realistic standard terms. This will help ensure that own funds and RWA divergences are correctly attributed to real differences in firms’ modeling and risk management practices and not differences that arise from margining assumptions used when the hypothetical portfolios are booked into systems.
For a second stage exercise, portfolios could for instance cover liquidity buffer securities held as available for sale in the banking book.
- In previous hypothetical portfolio exercises there have been differences in the Initial Market Valuation (IMV) used by firms for a number of instruments. This is usually due to the specified instruments/portfolios not having sufficient detail on all elements of the booking. When information is missing, firms must make assumptions to book trades in the systems. We would therefore recommend that Term Sheets be used to specify individual instruments at the start of the exercise as this will reduce the need for firms to make assumptions and reduce the risk of avoidable differences in the IMV.
Booking
- For all instruments, we highly recommend to use settlement dates that fall prior to the IMV reporting date in order to avoid comparison issues with other banks. Positions should be booked at a specific date ahead of the pre-validation exercise and no update should be required on the IMV reporting date in order to minimize the risk of operational errors.
IMV reporting time
- Point 1b mentions timing at 4.30pm London time. However, for most participating banks, the valuation timing in systems depends on the product. We therefore recommend the EBA leaves banks with flexibility in the timing of valuation and instead requires them to report the valuation timing they used for each instrument.
Annex VII.a: EBA market risk benchmark portfolios
- IR
o IR swaps for GBP, EUR, SEK, DKK and USD are not standard (variable legs indexed on 1 year LIBOR rates). As a result, swaptions written on them are bespoke. We would welcome clarification on the rationale for choosing non-standard swaps.
- Credit
o Point 1(i) mentions IMM dates should be used. We welcome clarification on whether this refers to IMM relative to the date of the exercise or relative to 21 Feb 2014.
o The description of CDS is not always homogeneous, sometimes it includes seniority, sometimes it doesn’t (although the RED code is always added, hence seniority is implicit). In order to avoid any doubt, we recommend being systematic by always providing seniority and RED Code.
o From September onwards there will be new ISDA definitions for CDSs. This might lead to confusion as to which products to book. If the EBA test portfolios are used for the 2014 exercise, then the EBA would need to give clear guidance.
o Note that for instrument #56 (AXA bond), the correct ISIN code is FR0011322668 and not FR001132266.
- Commodities
o For instrument #32 (crude oil puts), could the EBA give the exact reference of the month on which the put is written (example: WTI Dec-15 future which has a last trading date in Nov-15) and the month used for the strike determination (example: Jun-15 WTI future which has a last trading date in May-15)?
o For instruments #33 and #34, we seek clarification on the characteristics of the instruments: For instance, are they listed or OTC instruments? Are they ATM? Could the EBA provide specific dates to avoid any confusion?
- FX
o To avoid any differences between banks, we recommend the EBA defines trade details directly rather than by reference:
o Dates: it would be easier to have the exact maturity, rather than 3M.
o Strike: Exact strikes could be defined directly rather by reference to “the rate published by the ECB on 28 February 2014” or “the price corresponding to the three-month forward exchange rate as of end of day 21 February 2014”
o Additionally, we note some wording mistakes for instruments #30 and #31:
#30. 3-month short forward DKK/USD currency, (short long DKK, long short USD EUR) with 1 USD Million purchased at the DKK/USD reference rate published by the ECB on 28 February 2014
#31. 3-month short forward SEK/USD currency, (long USD, short EUR SEK) with 1 USD Million purchased at the SEK/USD reference rate published by the ECB on 28 February 2014
Sell call eur put usd with strike = Current FX Fwd x (1 + 1%) and sell put eur call usd with strike = Current FX Fwd x (1 - 1%)
We suggest the following process: banks select the portfolios which they think would be legitimate to exclude, based on a series of objective factors. These would be defined by the NCA and include quantitative factors (see below) as well as qualitative factors (e.g. portfolios in run-off). A discussion would then take place between the supervisor and the bank to reach a conclusion on whether it is appropriate to exclude such portfolios, or maintain them in the analysis.
Nevertheless, we do recognize that exemptions could lead to comparability issues for all-in portfolios.
Exemptions should include situations where the firm is under a model validation process for portfolios included in the samples and immaterial portfolios. A materiality threshold could be defined as:
- An absolute portfolio size and/or
- A relative portfolio size, in comparison to the total consolidated balance-sheet or to the balance-sheet size of the subsidiary.
We would happily help EBA in defining such materiality thresholds but we also believe that materiality could alternatively be defined in relative terms as per RTS on Model Change policy.
Portfolios with partial roll-out should also be exempted. Alternatively, the share of the portfolio under the SA should be highlighted and the bias resulting from different capital treatments eliminated. Moreover, as mentioned previously, any bias created from differing levels of defaulted assets should also be removed.
Exemptions should also include exceptional and temporary situations that are fully documented by the bank asking for the exemption, such as M&A situations impacting existing or new assets of the bank.
Lastly, local entities supervised by a host supervisor should also be exempted from solo reporting as long as their portfolios are included in the consolidated vision submitted to the home supervisor.
Q2. Do you consider that the benchmarks outlined in the RTS are sufficiently proportionate and flexible? Do you have any alternative benchmark proposals? If yes, please provide details.
Regardless of the appropriateness of the proposed benchmarks (discussed in answer to Q3), the question of proportionality and flexibility is closely related to the obligations imposed on NCAs when a contribution is deemed to be an “outlier” when compared to the defined benchmarks.To the extent that NCAs are expected to investigate any “outlier” and provide the EBA with feedback, we believe that the proposed benchmarks are overly prescriptive and disproportionate. For the sake of illustration, the “1st and 4th quartile” rule would systematically require competent authorities to investigate half of the contributions. At least a more selective distribution of fractiles (e.g. 5% lowest and 5% highest values), or asymmetric fractiles (e.g. 10% lowest, and 2% highest) could be contemplated as triggers for abnormal calibrations. Similarly, the “80% SA output criteria” is likely to result in most contributions being considered as outliers for market risk test portfolios with diversification/netting features.
Moreover, as the EBA is going to generate the results of the benchmarking based on NCA conclusions (Article 2, RTS), the criteria used by the different NCAs must be comparable. This comparability must be assured by the EBA through a more detailed specification (i.e. more detailed than in the current RTS).
It is also worth mentioning that the benefits of diversification (portfolio and geographical diversification) should be considered within the benchmarking assessment.
Alternative proposals are discussed in answers to questions 3 and 4 below.
Q3. What limitations do you see in relation to the use of the proposed benchmarks, i.e., (i) first and the fourth quartiles; (ii) comparison between own funds under the internal models and the standardised approach; and (iii) comparison between estimates and outturns?
(i) First and fourth quartilesWe recognize the importance of using common and objective benchmarks to identify extreme values. However, we consider the current criterion (1st and 4th quartile irrespective of the dispersion and clustering of the contributions) unsuitable. This is because it can lead to the identification of a contributor being “mechanically” an outlier while, by any reasonable materiality standard, it would not be considered so. For example, in cases of low dispersion where all the contributed results are very close to each other (say, within 1% of each other), 50% of the contributors based on the 1st and 4th quartile rule will still be identified as outliers. Take another example where all contributors have results that are very close together, except for one that is much lower than the others. The 1st and 4th quartile rule would correctly include this latter contributor as an outlier, but it will also include contributors on the high side even though their results might just be marginally higher than most of the firms. To summarise, there is in our view no need to identify outliers when the dispersion is tight and there is no reason why the number of outliers on both side of the distribution should be equal.
Not only would the identification of many these types of false outliers be burdensome for both the regulators and the firms, the 1st and 4th quartile rule can also potentially lead to instability as whoever is identified as a outlier on the downside would want to use a model that generates more conservative results to exit that status. By doing so, next time around, it will not appear as an outlier, while another firm who was previously just within the quartile cutoff might then be pushed out even though it will not have done anything differently.
To overcome these issues, we propose to thin out the percentile range for indentifying potential outliers on top of which we would overlay a materiality rule. Specifically, as a first step, we would identify the top 10% and bottom 10% as potential outliers. Secondly, we would then check if their results are more than x% away (say 20%) from the median. Only those contributors with results more than x% away from the median would ultimately be retained as outliers.
The advantage of this modified rule is that it takes into account both relative ordering and absolute differences. If the dispersion is very tight and no contributor has results more than x% away from the median, than there will be no outlier. However, if there is only one outlier far away from everyone else, it will have been correctly identified without any other contributors inadvertently being called outliers when in fact they are not.
(ii) Comparison between own funds under the internal models and the standardized approach
As explained in our answer to question Q1, our view is that it is not appropriate to define a common reference level of own funds at test portfolio level. Moreover, comparing required own funds calculated under an internal model and the SA could be misleading due to inherent simplifications, omissions and approximations in the parameterization of the SA where sound risk management practices (including business strategies, recovery policies, the presence of collateral, the use of hedging etc.) are not taken into account. By way of example, within the course of the Phase 2 SIG TB market risk benchmarking exercise, CRM floor contributions were even more volatile than CRM contributions themselves.
In addition to being risk insensitive, SA metrics are not universal standards in their implementation across jurisdictions. Not only could the interpretation of the same standards differ across banks, a number of factors under this approach are also open to national supervisory discretion. For instance, the treatment of sovereign risk under the SA is subject to national discretion and a lower risk weight can be applied to banks’ exposures to the sovereign of incorporation denominated in domestic currency and funded in that currency.
We therefore believe that the appropriate level of capital for a given portfolio is firm-specific and must be assessed in the light of the firm’s ability to risk-manage such a portfolio, the performance of the internal model and the representativeness of the portfolio with respect to the overall internal model.As far as market risk is concerned, should the EBA ultimately choose to maintain SA metrics as a common benchmark, we urge the EBA to consider at a minimum the following:
o The application of SA output as a benchmark should be postponed until the new SA under the Fundamental Review of the Trading Book (FRTB) is implemented for market risk,
o The computation of SA metrics should be performed at EBA level and the methodology used made available to all participating banks. This would be particularly useful for certain specific portfolios where national implementation of SA metrics diverges.
(iii) Comparison between estimates and outturns
Before comparing estimates and outturns, one must assess whether they refer to comparable economic situations. We consider that it is important to compare regulatory parameters (point-in-time, through-the-cycle or downturn) with outturns from the current situation to assess if banks from the same country are equally affected by the business cycle. Such a comparison is a way to assess differences in the dynamics of parameter estimates.
Furthermore, the EBA should give precise technical guidance on the computation of outturns (for instance in the form of a spreadsheet), and these computations should take into account all components of the statistical validity of risk estimates, such as the number of years of available history, default sample size, number of models in use for the respective cluster, etc. This guidance should be developed in consultation with the industry to ensure its suitability and that it can be replicated for institutions across Member States.
Consider for example a portfolio where origination has decreased over past years due to prudent risk management. Perhaps counterintuitively, the cost of risk on the portfolio is likely to have risen over a 5-year history (as the number of contracts in the portfolio will have decreased). However, the relative information within the 5 individual, annual default rates is not identical due to the change in risk profile and risk appetite. We would welcome guidance on how should such changes should be reflected in confidence intervals.
Additionally, with respect to credit risk, the use of the 97.5% confidence level is, in our opinion, a new paradigm which may prove inappropriate on some portfolios (for instance owing to various sample sizes or the weight of expert judgment in the rating process). We are not aware of economic or academic studies proving the importance or relevance of such a confidence level.
Finally we believe that competent authorities should not consider the different criteria as single “show-stoppers” but instead they should base their analysis on the combination of their outcome.
Q4. What in your view is the most appropriate benchmark and/or approach for the assessment of the level of potential underestimation of own funds requirements?
Generally, we believe that a comparison between groups of exposures according to certain key drivers is a good idea. In this context, we note that Art 78 imposes the responsibility for detecting outliers and explaining differences on the NCAs. We consider that such drivers have to be established by the EBA and, to the extent possible, need to be the same for all NCAs in order to ensure comparability of the results across different jurisdictions In our view, the EBA should also assume a more active role in international benchmarking by coordinating with other international standard-setters and supervisory authorities.We consider that the only truly robust and valuable approach to identifying the potential under-estimation of own funds requirements is one based on the comparison of model-estimated values and realized values, i.e. backtesting. Nevertheless, NCAs should also use backtesting in conjunction with other sources of investigation like modeling construction and analysis as well as validation history (local supervisory practices and decisions on particular models). Lastly, we wish to stress that benchmarking can never be a substitute for NCAs exercising expert judgment.
Market risk
In order to help identify RWA variance drivers, our proposal is to subject market risk test portfolios to backtesting consistently with the Basel FRTB approach (which introduces backtesting at a much more granular level).
Consistently with its mandate, the EBA’s work should be annual and aligned with the timeline of the Basel monitoring exercises.
Credit risk
Lending practices are firm specific and model output and losses will also therefore be firm specific. The application of different accounting frameworks and a firms’ accounting policies will also mean that the measurement of losses for the same exposure may be different. It is important that the assessment of firms’ responses factors in these differences.
Along the same lines, in some cases NCAs have introduced floors or restrictions on the parameters that firms may use in internal model approaches. Some of these floors may be applied across all firms within a Member State, whilst other floors may be applied on a firm-by-firm basis. We would recommend that the EBA controls for such differences otherwise they may incorrectly be attributed to firms’ models and skew the results of the exercise.
Additionally, to avoid creating false outliers, expected losses (%) or RW (%) rather than PDs/LGDs should be the first point of comparison. Otherwise, two banks, one with a very early default definition and the other with a very late one, will tend to have high PD/low LGD and (respectively) low PD/high LGD whereas the results in EL% or RW% could be very similar.
We understand the assessment of internal models is one of the objectives of the “General provisions” laid out in Article 8 of the RTS. Nevertheless we do not fully understand to what extent the NCAs will make use of the information requested in Article 8 paras 1 and 2 of the RTS and would welcome clarification that the EBA will use this information to validate firm specific outcomes.
To conclude this section, we wish to stress that benchmarking exercises of banks’ risk assessment must be performed taking model performance into account and by backtesting outputs. It must be well understood that being an outlier in the context of the RWA benchmarking exercises does not necessarily imply that models are not performing appropriately. In fact, a well-performing model means that a bank is doing a good job in predicting its risks and assessing its capital, but it can still produce outlier results in RWA benchmarking exercises.
Q5. Which set of market risk portfolios do you consider more appropriate for the initial exercise conducted under Article 78?
For the initial 2014 initial exercise, we highly recommend the use of the BCBS test portfolios. Indeed, we learnt from previous market risk benchmarking exercises that proper portfolio specification, consistent booking across participants and a pre-validation exercise are key steps to ensure the exercise meets its targets.Given other ongoing regulatory exercises with overlapping timelines that many market risk experts have been focusing on (e.g. FRTB QIS, BCBS/EBA joint counterparty risk and CVA benchmarking exercise to mention only two such examples) and the very tight timeframe and close kick-off date contemplated by the EBA for the 2014 exercise, we consider that the conditions are not met to ensure the 2014 exercise can be properly performed on the basis of a new set of portfolios
Q6. As explained in the background section, do you consider the approach proposed by the EBA appropriate for future annual exercises?
Market RiskGiven the tight time frame referred to above, we favor the use of HPE portfolios for the first exercise of Q4 2014. Starting from the 2015 exercise, we could then switch to the new portfolios proposed by the EBA.
Moreover, we fully support the EBA’s proposition to organize a pre-validation phase to ensure correct booking of instruments. Before the start of any exercise, NCAs should ensure that initial market valuations are within an acceptable range and questions concerning instructions are resolved. Previous exercises such as the FRTB QIS on HPE showed that this pre-validation phase is of key importance to limit the risk of operational errors.
Lastly, we want to draw the EBA’s attention on the necessity for banks to receive portfolios specifications well in advance of the exercise. Booking positions in test portfolios, checking them and performing process validation requires time which cannot be reduced. Multiple iterations of instructions should be avoided to the extent possible to limit confusion or late changes to the portfolios.
Credit Risk
We note that previous public RWA benchmarking exercises have in particular highlighted the impact of banks' risk profiles and defaulted assets as being amongst the key drivers behind RWA variability. When benchmarking institutions, the effect of defaulted assets on a bank’s global RWA should be controlled or neutralized because, all else being equal, two banks with two different amounts of defaulted assets in their balance sheets will show two diverging RWA.
The benchmarking analysis should also take into account the type of banks being compared: a universal bank and a pure CIB bank may have legitimate divergent assessments of the RWA of the same assets, due to their different specializations.
This being said, we support the approach suggested by the EBA to alternate benchmarking exercises on LDP and HDP. This is because a thorough examination of the results, followed by explanations between supervisors and banks will obviously be a lengthy process, and (at least on the credit risk side) loss trends emerge progressively: macroeconomic changes are the main risk drivers, and will take time to materialize in loss parameters.
The HPE approach for LGD seems to be particularly burdensome to implement, especially as far as secured LGD is concerned, for at least three main reasons:
- This hypothetical exercise would be a true hypothetical exercise (compared to previous HPE exercises that have relied on existing information), since the underlying transactions would not exist. Therefore, the reliability of estimates would be questionable and the workload for banks much higher.
- It is unlikely that banks will be able to provide all the requested information to determine CCF/LGD Secured. LGD Secured is not just a simple function of the collateral only, it also depends on many different factors, including, obviously, the collateral (type, location, condition, etc.) but also the ability to have access to it (seniority of the claim, legal environment, nature of the counterparty etc.), the characteristic of the loan (type, LTV etc.), the usage of the asset, etc.
- The outputs would also depend on the area of expertise of some institutions: for instance, a German bank would be able to give an LGD estimate for a German real estate transaction, whereas it would probably be much more difficult for a retail Spanish bank to give an LGD for a German transaction.
The concept of cluster appears to be a good tool to tackle issues of representativeness and comparability of data collected across banks in the benchmarking process. Nevertheless, clusters should be defined pragmatically along drivers that are applicable to the majority of banks and with appropriate thresholds of materiality. Currently, we are concerned that the level of granularity set out in the EBA’s proposals will produce immaterial and unusable clusters.
As for HDP, we consider that the portfolios defined for mortgages are far too granular:
- Until banks set up/adapt an adequate IT solution for reporting for purposes of the benchmarking exercise, all of the requested information may not be directly available from all firms’ current IT systems (LTV buckets for instance might be recorded in another business or IT management environment).
- By selecting too many, excessively granular clusters, especially on retail exposures, comparability might be difficult or in vain, all the more so that these portfolios are more sensitive to local specificities.
- Very small clusters would not fit with the range of application of banks’ internal models: for instance one cluster could be covered by 2 models which makes backtesting difficult to implement and interpret.
Counterparty Risk
We reiterate that the consultation paper contains little information on the proposed counterparty risk portfolios that would be used for future annual exercises.
Q7. Do you have any alternative proposals? If yes, please provide details.
We take the opportunity here to recall that any alternative benchmarks should always respect the following principles:- The first step of any benchmarking exercise should be to establish what the key drivers behind RWA divergences are.
- Benchmarks should be expressed in terms of ranges of acceptable divergences rather than single, simplistic values.
- Benchmarking frameworks should be refined by segmentation according to the relevant key drivers of divergences such as, but not limited to, bank type, asset classes, geographical sector, etc.
Q8. Which of the two options for phasing-in do you consider preferable?
We are in favor of option 2, provided that there is no significant evolution on portfolios except what could improve the relevance of these portfolios.Q9. Do you see any potential ambiguities in the credit risk portfolios defined in Annex I? Please identify the relevant portfolio providing details and any suggestions that would eliminate these ambiguities.
The industry believes that the best way to deal with any ambiguities is through a targeted Q & A process. As an ongoing process for regular benchmarking exercises, it would provide the most flexible and comprehensive platform within which the potential evolution of portfolios and models can be discussed between banks and the EBA.Nevertheless, we would already like to provide the following comments/questions:
Annex I
- All: Col 060 - We assume that banks would be required to provide the PDs behind the rating scale to support the backtesting.
- All: Col 490 - We consider secured categories to include exposures that are partially secured to align with LGD modeling for single exposures.
- C101 - Portfolio Name & Sector do not seem to be always compatible: e.g. in the “Large Corporate Sample” portfolio, sectors like “Credit institutions” should not be allowed.
- C101: Col 450 - We consider country of residency to be country of operation.
- C102 / C103: Col 130 = Type of collateral is a mixture of collateral and personal guarantees. It is possible to have a personal guarantee or a credit derivative while having physical or financial collateral. Our proposal is to separate collateral and guarantees.
- C102 / C104: Col 270 = facilities like credit lines or credit cards may have both on balance and off balance exposures.
- C103: Col 020 - The templates are incomplete as they do not include Retail Other and Retail QRRE portfolios.
- C103: Col 160 - It is not appropriate to provide maturity for Retail Portfolios as it is not used to calculate capital requirements.
- C103: Col 450: - Annex I considers different countries inside the EU. Annex II mentions only EU/Non-EU options.
- C103: Col 460 - The definition of construction and the rationale for requesting a separate cluster for it and not for other lending types is not clear. For example, it would not capture all forms of lending secured on real estate.
- C104: Col 130 = Seems to be a mixture of real collateral, credit protection and seniority. In our opinion those are different dimensions and should be treated separately. Additionally, In our view negative pledges are not always enforced and / or included in LGD modeling so do not effect recoveries. This should be removed from the benchmarking exercises.
- C104: Col
Annex V
C105: Col 630 - requires firms to provide Long-run PD but this is not used by all institutions - for example Point-in Time PD models
Counterparty credit risk
We note that the EBA proposes to align with the Basel SIG TB exercise for the benchmarking of counterparty credit risk in 2014 to cover IMM and CVA models. We are concerned that the EBA paper contains little information on the proposed counterparty risk portfolios for 2014 or on what portfolios would be used from 2015.
It is also not clear what standards of assessment or reporting will be applied for counterparty risk and CVA as these are not detailed in the consultation paper and more specifically in neither the RTS or ITS. For example, for CVA, it is not clear whether the exercise will be required to capture both advanced and standardized calculations, or whether similar requirements for market risk are relevant, such as initial market valuations.
The EBA paper refers to the ‘scarcity of resources and the existing workload both for banks and NCAs. We would agree with EBA’s concern about resources and would propose that the 2014 exercise cover only the IMM component of counterparty risk and not CVA. Further, we would recommend that only one asset class be covered in the first year. Benchmarking interest rate products, for example, would cover a significant amount of the exposure and would allow for a meaningful comparison to be made between firms. This would be analogous to the phased approach for credit risk supported by the Associations (i.e. where EBA propose to perform the analysis of High Default Portfolios (HDP) and Low Default Portfolios (LDP) in alternate year).
We note that the counterparty risk sections in Annexes VIII and IX are incomplete. When the EBA is able to publish details on the counterparty risk benchmarking proposals, we would recommend that the EBA considers defining consistent margin terms and collateralisation agreements. Our members would be happy to help define realistic standard terms. This will help ensure that own funds and RWA divergences are correctly attributed to real differences in firms’ modeling and risk management practices and not differences that arise from margining assumptions used when the hypothetical portfolios are booked into systems.
Q10. Do you have any suggestions for additional credit risk portfolios? Please provide details.
We support the EBA’s proposal to focus on vanilla credit instruments, since comparability is likely to be difficult to achieve on more sophisticated exposures or on exposures held by a limited number of contributing banks.For a second stage exercise, portfolios could for instance cover liquidity buffer securities held as available for sale in the banking book.
Q11. Do you see any potential ambiguities in the market risk portfolios defined in Annexes VII.a and VII.b? Please identify the relevant portfolio providing details and any suggestions that would eliminate these.
Term Sheets- In previous hypothetical portfolio exercises there have been differences in the Initial Market Valuation (IMV) used by firms for a number of instruments. This is usually due to the specified instruments/portfolios not having sufficient detail on all elements of the booking. When information is missing, firms must make assumptions to book trades in the systems. We would therefore recommend that Term Sheets be used to specify individual instruments at the start of the exercise as this will reduce the need for firms to make assumptions and reduce the risk of avoidable differences in the IMV.
Booking
- For all instruments, we highly recommend to use settlement dates that fall prior to the IMV reporting date in order to avoid comparison issues with other banks. Positions should be booked at a specific date ahead of the pre-validation exercise and no update should be required on the IMV reporting date in order to minimize the risk of operational errors.
IMV reporting time
- Point 1b mentions timing at 4.30pm London time. However, for most participating banks, the valuation timing in systems depends on the product. We therefore recommend the EBA leaves banks with flexibility in the timing of valuation and instead requires them to report the valuation timing they used for each instrument.
Annex VII.a: EBA market risk benchmark portfolios
- IR
o IR swaps for GBP, EUR, SEK, DKK and USD are not standard (variable legs indexed on 1 year LIBOR rates). As a result, swaptions written on them are bespoke. We would welcome clarification on the rationale for choosing non-standard swaps.
- Credit
o Point 1(i) mentions IMM dates should be used. We welcome clarification on whether this refers to IMM relative to the date of the exercise or relative to 21 Feb 2014.
o The description of CDS is not always homogeneous, sometimes it includes seniority, sometimes it doesn’t (although the RED code is always added, hence seniority is implicit). In order to avoid any doubt, we recommend being systematic by always providing seniority and RED Code.
o From September onwards there will be new ISDA definitions for CDSs. This might lead to confusion as to which products to book. If the EBA test portfolios are used for the 2014 exercise, then the EBA would need to give clear guidance.
o Note that for instrument #56 (AXA bond), the correct ISIN code is FR0011322668 and not FR001132266.
- Commodities
o For instrument #32 (crude oil puts), could the EBA give the exact reference of the month on which the put is written (example: WTI Dec-15 future which has a last trading date in Nov-15) and the month used for the strike determination (example: Jun-15 WTI future which has a last trading date in May-15)?
o For instruments #33 and #34, we seek clarification on the characteristics of the instruments: For instance, are they listed or OTC instruments? Are they ATM? Could the EBA provide specific dates to avoid any confusion?
- FX
o To avoid any differences between banks, we recommend the EBA defines trade details directly rather than by reference:
o Dates: it would be easier to have the exact maturity, rather than 3M.
o Strike: Exact strikes could be defined directly rather by reference to “the rate published by the ECB on 28 February 2014” or “the price corresponding to the three-month forward exchange rate as of end of day 21 February 2014”
o Additionally, we note some wording mistakes for instruments #30 and #31:
#30. 3-month short forward DKK/USD currency, (short long DKK, long short USD EUR) with 1 USD Million purchased at the DKK/USD reference rate published by the ECB on 28 February 2014
#31. 3-month short forward SEK/USD currency, (long USD, short EUR SEK) with 1 USD Million purchased at the SEK/USD reference rate published by the ECB on 28 February 2014
Q12. Do you have any suggestions for additional market risk portfolios? Please provide details.
For FX test portfolios, we propose to introduce a FX vanilla option out of the money with a strike far from the FX forward by 2.33 standard-deviations. The goal is to check if the VaR captures the convexity between the current FX Spot and the 99% FX Spot bump: does the methodology use a Taylor approach or a full Revaluation? For the sake of illustration, the related strategy could be:Sell call eur put usd with strike = Current FX Fwd x (1 + 1%) and sell put eur call usd with strike = Current FX Fwd x (1 - 1%)
Q13 Do you agree with the possibility of allowing firms to refrain from reporting portfolios if one of the conditions stated in Article 3 is met?
We indeed think some exemptions should be granted, especially when they contribute to a better understanding of the results overall (in practice, adding minor portfolios at the expense of the overall precision of the results would be undesirable).We suggest the following process: banks select the portfolios which they think would be legitimate to exclude, based on a series of objective factors. These would be defined by the NCA and include quantitative factors (see below) as well as qualitative factors (e.g. portfolios in run-off). A discussion would then take place between the supervisor and the bank to reach a conclusion on whether it is appropriate to exclude such portfolios, or maintain them in the analysis.
Nevertheless, we do recognize that exemptions could lead to comparability issues for all-in portfolios.
Q14 Do you have any suggestion about additional exemptions from reporting? If yes, please provide details.
We would also welcome similar exemptions market and credit risk:Exemptions should include situations where the firm is under a model validation process for portfolios included in the samples and immaterial portfolios. A materiality threshold could be defined as:
- An absolute portfolio size and/or
- A relative portfolio size, in comparison to the total consolidated balance-sheet or to the balance-sheet size of the subsidiary.
We would happily help EBA in defining such materiality thresholds but we also believe that materiality could alternatively be defined in relative terms as per RTS on Model Change policy.
Portfolios with partial roll-out should also be exempted. Alternatively, the share of the portfolio under the SA should be highlighted and the bias resulting from different capital treatments eliminated. Moreover, as mentioned previously, any bias created from differing levels of defaulted assets should also be removed.
Exemptions should also include exceptional and temporary situations that are fully documented by the bank asking for the exemption, such as M&A situations impacting existing or new assets of the bank.
Lastly, local entities supervised by a host supervisor should also be exempted from solo reporting as long as their portfolios are included in the consolidated vision submitted to the home supervisor.