Skip to main content
Log in

When is it Effective to Focus on the Alliance? Analysis of a Within-Client Moderator

  • Original Article
  • Published:
Cognitive Therapy and Research Aims and scope Submit manuscript

Abstract

Contemporary studies on the working alliance seek to move forward from demonstrating an association between alliance and outcome to investigating how alliance can be used to maximize treatment outcome by identifying the clients for whom state-like changes in alliance predict symptomatic change (between-clients moderators). Yet, very little is known empirically on when state-like changes in alliance predict outcome for individual clients (within-client moderators). The present study, based on a sample of 327 clients, demonstrates that state-like changes in alliance at a given session have a significant effect on subsequent session outcome only in the case of higher life satisfaction in that session. This finding suggests that strengthening in the state-like component of the alliance has a greater effect on outcome when the client suffers less from poor life satisfaction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

Notes

  1. Various studies have used different models to examine this association, some showing that stronger alliance at a given time point predicted less severe symptoms at a subsequent time point, while controlling for previous symptomatic levels (e.g., Zilcha-Mano et al. 2014). Other studies used slightly different models, showing that patients who reported improvement in alliance over their expected level, are more likely to report greater reduction of symptoms, while controlling for previous symptomatic levels (e.g., Falkenström et al. 2013; Zilcha-Mano and Errázuriz 2015).

  2. It is important to stress that this value, although standardized, is by definition not comparable to other values taken from models that are not identical to the reported one (e.g., without log of time as both a fixed and a random effect, etc.).

  3. We repeated the analysis using a model that controlled for concurrent (time T) symptoms and prior change in symptoms (i.e., symptom change occurring before time T) as well as for log of time. This model revealed a significant interaction between within-client alliance and life satisfaction at the previous session in predicting outcome (β = −0.09, SE = 0.04, p = .02).

  4. To explore the question whether this moderating effect is specific to CBT, we performed a three-way interaction between within-client alliance and life-satisfaction at the last session with treatment condition (CBT and non-CBT therapists) to predict outcome for the entire sample (N = 547, CBT and non-CBT therapists). This interaction was not significant (F (1,2592) = 0.55, p = .46).

References

  • American Psychiatric Association (2000). Diagnostic criteria from DSM-IV-TR. Washington, D.C.: American Psychiatric Association.

    Google Scholar 

  • Asar, Ö., Ritchie, J., Kalra, P. A., & Diggle, P. J. (2015). Joint modelling of repeated measurement and time-to-event data: an introductory tutorial. International Journal of Epidemiology, 44(1), 334–344.

    Article  PubMed  Google Scholar 

  • Baldwin, S. A., & Imel, Z. E. (2013). Therapist effects: Findings and methods. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 258–297). New York: Wiley.

    Google Scholar 

  • Baldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007). Untangling the alliance-outcome correlation: Exploring the relative importance of therapist and patient variability in the alliance. Journal of Consulting and Clinical Psychology, 75, 842–852. doi:10.1037/0022-006X.75.6.842.

    Article  PubMed  Google Scholar 

  • Barber, J. P. (2009). Toward a working through of some core conflicts in psychotherapy research. Psychotherapy Research, 19(1), 1–12.

    Article  PubMed  Google Scholar 

  • Beck, A. T., Rush, A. J., Shaw, B. F., & Emery, G. (1979). Cognitive therapy of depression. New York, NY: Guilford Publications.

    Google Scholar 

  • Bjørnskov, C. (2010). How comparable are the Gallup World Poll life satisfaction data? Journal of Happiness Studies, 11(1), 41–60.

    Article  Google Scholar 

  • Bjørnskov, C., Dreher, A., & Fischer, J. A. (2010). Formal institutions and subjective well-being: Revisiting the cross-country evidence. European Journal of Political Economy, 26(4), 419–430.

    Article  Google Scholar 

  • Bolger, N., & Laurenceau, J. P. (2013). Intensive longitudinal methods. New York, NY: Guilford.

    Google Scholar 

  • Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research & Practice, 16(3), 252–260.

    Article  Google Scholar 

  • Braun, J. D., Strunk, D. R., Sasso, K. E., & Cooper, A. A. (2015). Therapist use of Socratic questioning predicts session-to-session symptom change in cognitive therapy for depression. Behaviour Research and Therapy, 70, 32–37.

    Article  PubMed  PubMed Central  Google Scholar 

  • Castonguay, L. G., Constantino, M. J., McAleavey, A. A., & Goldfried, M. R. (2010). The therapeutic alliance in cognitive-behavioral therapy. In J. C. Muran & J. P. Barber (Eds.), The Therapeutic alliance: An evidence-based guide to practice (pp. 150–171). New York, NY: Guilford Press.

    Google Scholar 

  • Castonguay, L. G., & Hill, C. E. (2012). Transformation in psychotherapy: Corrective experiences across cognitive behavioral, humanistic, and psychodynamic approaches. Washington, DC: American Psychological Association.

    Book  Google Scholar 

  • Castonguay, L. G., Schut, A. J., Aikens, D. E., Constantino, M. J., Laurenceau, J. P., Bologh, L., & Burns, D. D. (2004). Integrative cognitive therapy for depression: A preliminary investigation. Journal of Psychotherapy Integration, 14(1), 4.

    Article  Google Scholar 

  • Cheavens, J. S., Strunk, D. R., Lazarus, S., & Goldstein, L. A. (2012). Cognitive behavioral approaches to the treatment of depression: A preliminary test of the compensation and capitalization models. Behaviour Research and Therapy, 50, 699–706.

    Article  PubMed  Google Scholar 

  • Cohen, J. S., Edmunds, J. M., Brodman, D. M., Benjamin, C. L., & Kendall, P. C. (2013). Using self-monitoring: Implementation of collaborative empiricism in cognitive-behavioral therapy. Cognitive and Behavioral Practice, 20(4), 419–428.

    Article  Google Scholar 

  • Cole, D. A., & Maxwell, S. E. (2003). Testing mediational models with longitudinal data: questions and tips in the use of structural equation modeling. Journal of Abnormal Psychology, 112(4), 558.

    Article  PubMed  Google Scholar 

  • Connors, G. J., Maisto, S. A., Schlauch, R. C., Dearing, R. L., Prince, M. A., & Duerr, M. R. (2016). Therapeutic alliances predict session by session drinking behavior in the treatment of alcohol use disorders. Journal of Consulting and Clinical Psychology, 84(11), 972–982.

    Article  PubMed  PubMed Central  Google Scholar 

  • Crits-Christoph, P., Gibbons, M. B. C., Crits-Christoph, K., Narducci, J., Schamberger, M., & Gallop, R. (2006). Can therapists be trained to improve their alliances? A preliminary study of alliance-fostering psychotherapy. Psychotherapy Research, 16(03), 268–281.

    Article  Google Scholar 

  • Crits-Christoph, P., Gibbons, M. B. C., Hamilton, J., Ring-Kurtz, S., & Gallop, R. (2011). The dependability of alliance assessments: The alliance–outcome correlation is larger than you might think. Journal of Consulting and Clinical Psychology, 79(3), 267.

    Article  PubMed  PubMed Central  Google Scholar 

  • Crits-Christoph, P., Gibbons, M. B. C., & Mukherjee, D. (2013). Psychotherapy process-outcome research. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (pp. 298–340). New York NY: John Wiley & Sons.

    Google Scholar 

  • Cronin, T. J., Lawrence, K. A., Taylor, K., Norton, P. J., & Kazantzis, N. (2015). Integrating between-session interventions (homework) in therapy: The importance of the therapeutic relationship and cognitive case conceptualization. Journal of Clinical Psychology, 71(5), 439–450.

    Article  PubMed  Google Scholar 

  • Curran, P. J., & Bauer, D. J. (2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology, 62, 583–619.

    Article  PubMed  PubMed Central  Google Scholar 

  • DeRubeis, R. J., Brotman, M. A., & Gibbons, C. J. (2005). A conceptual and methodological analysis of the nonspecifics argument. Clinical Psychology: Science and Practice, 12(2), 174–183.

    Google Scholar 

  • DeRubeis, R. J., Cohen, Z. D., Forand, N. R., Fournier, J. C., Gelfand, L. A., & Lorenzo-Luaces, L. (2014). The Personalized Advantage Index: Translating research on prediction into individualized treatment recommendations. A demonstration. PLoS One, 9(1), e83875.

    Article  PubMed  PubMed Central  Google Scholar 

  • DeRubeis, R. J., Evans, M. D., Hollon, S. D., Garvey, M. J., Grove, W. M., & Tuason, V. B. (1990). How does cognitive therapy work? Cognitive change and symptom change in cognitive therapy and pharmacotherapy for depression. Journal of Consulting and Clinical Psychology, 58(6), 862.

    Article  CAS  PubMed  Google Scholar 

  • Diefenbach, G. J., Abramowitz, J. S., Norberg, M. M., & Tolin, D. F. (2007). Changes in quality of life following cognitive–behavioral therapy of obsessive–compulsive disorder. Behavioral Research and Therapy, 45, 3060–3068.

    Article  Google Scholar 

  • Diener, E., Kahneman, D., & Helliwell, J. (2010). International differences in well-being. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Dobson, D. J., & Dobson, K. S. (2013). In-session structure and collaborative empiricism. Cognitive and Behavioral Practice, 20(4), 410–418.

    Article  Google Scholar 

  • Dobson, K. S. (Ed.). (2010). Handbook of cognitive behavioral therapy. New York, NY: The Guilford Press.

    Google Scholar 

  • DuPont, R. L., Rice, D. P., Miller, L. S., Shiraki, S. S., Rowland, C. R., Harwood, H. J. (1996). Economic costs of anxiety disorders. Anxiety, 2, 167–172.

    Article  CAS  PubMed  Google Scholar 

  • Eng, W., Coles, M. E., Heimberg, R. G., & Safren, S. A. (2005). Domains of life satisfaction in social anxiety disorder: Relation to symptoms and response to cognitive-behavioral therapy. Journal of Anxiety Disorders, 19(2), 143–156.

    Article  PubMed  Google Scholar 

  • Errázuriz, P., Zilcha-Mano, S., & Calvo, E. (2017a). To know or not to know: Does therapists access to feedback about patients psychological dysfunction and alliance perception affect treatment adherence, alliance, and outcome? Manuscript under review.

  • Errázuriz, P., Opazo, S., Behn, A., Silva, O., & Gloger, S. (2017b). Spanish adaptation and validation of the outcome questionnaire OQ-30.2. Frontiers in Psychology, 8, 673.

    Article  PubMed  PubMed Central  Google Scholar 

  • Falkenström, F., Ekeblad, A., & Holmqvist, R. (2016). Improvement of the working alliance in one treatment session predicts improvement of depressive symptoms by the next session. Journal of Consulting and Clinical Psychology, 84(8), 738–751. doi:10.1037/ccp0000119.

    Article  PubMed  Google Scholar 

  • Falkenström, F., Granström, F., & Holmqvist, R. (2013). Therapeutic alliance predicts symptomatic improvement session by session. Journal of Counseling Psychology, 60(3), 317. 10.1037/a0032258.

    Article  PubMed  Google Scholar 

  • Fleche, S., Smith, C., & Sorsa, P. (2012). Exploring Determinants of Subjective Wellbeing in OECD Countries: Evidence from the World Value Survey, OECD Statistics Working Papers, 2012/01, OECD Publishing. doi:10.1787/5k9ffc6p1rvben.

  • Flückiger, C., Del Re, A. C., Horvath, A. O., Symonds, D., Ackert, M., & Wampold, B. E. (2013). Substance use disorders and racial/ethnic minorities matter: A meta-analytic examination of the relation between alliance and outcome. Journal of Counseling Psychology. doi:10.1037/a0033161.

    PubMed  Google Scholar 

  • Freeman, A., & McCloskey, R. D. (2003). Impediments to effective psychotherapy. In R. L. Leahy (Ed.), Roadblocks in cognitive-behavioral therapy: Transforming challenges into opportunities for change (pp. 24–48). New York: Guilford Press.

  • Garratt, G., Ingram, R. E., Rand, K. L., & Sawalani, G. (2007). Cognitive processes in cognitive therapy: Evaluation of the mechanisms of change in the treatment of depression. Clinical Psychology: Science and Practice, 14(3), 224–239.

    Google Scholar 

  • Hatcher, R. L., & Barends, A. W. (2006). How a return to theory could help alliance research. Psychotherapy: Theory, Research, Practice, Training, 43(3), 292.

    Article  Google Scholar 

  • Hoffart, A. (2016). Cognitive models for panic disorder with agoraphobia: A study of disaggregated within person effects. Journal of Consulting and Clinical Psychology, 84(9), 839–844. doi:10.1037/ccp0000114.

    Article  PubMed  Google Scholar 

  • Hofmann, S. G., Wu, J. Q., & Boettcher, H. (2014). Effect of cognitive-behavioral therapy for anxiety disorders on quality of life: a meta-analysis. Journal of Consulting and Clinical Psychology, 82, 375–391. doi:10.1037/a0035491.

    Article  PubMed  PubMed Central  Google Scholar 

  • Horvath, A. O., Del Re, A., Flückiger, C., & Symonds, D. (2011). Alliance in individual psychotherapy. Psychotherapy, 48, 9–16. doi:10.1037/a0022186.

    Article  PubMed  Google Scholar 

  • IsHak, W. W., Greenberg, J. M., Balayan, K., Kapitanski, N., Jeffrey, J., Fathy, H., ... Rapaport, M. H., et al. (2011). Quality of life: The ultimate outcome measure of interventions in major depressive disorder. Harvard Review of Psychiatry, 19(5), 229–239.

    Article  PubMed  Google Scholar 

  • Kazantzis, N., Cronin, T. J., Dattilio, F. M., & Dobson, K. S. (2013). Introduction: Using techniques via the therapeutic relationship. Cognitive and Behavioral Practice, 20(4), 385–389.

    Article  Google Scholar 

  • Kolovos, S., Kleiboer, A., &Cuijpers, P (2016). The effect of psychotherapy for depression on quality of life: A meta-analysis. British Journal of Psychiatry, 209, 1–9.

  • Kuyken, W., Orley, J., Power, M., Herrman, H., Schofield, H., Murphy, B., et al. (1995). The World Health Organization Quality of Life Assessment (WHOQOL): Position paper from the World Health Organization. Social Science & Medicine, 41, 1403–1409.

    Article  Google Scholar 

  • Lambert, M. J., Vermeersch, D. A., & Brown, G. J. (2004). Administration and scoring manual for the OQ-30.2. East Setauket. NY: American Professional Credentialing Services.

    Google Scholar 

  • Leahy, R. L. (2003). Emotional schemas and resistance. In Roadblocks in cognitive-behavioral therapy: Transforming challenges into opportunities for change (pp. 91–115).

  • Littell, R. C., Milliken, G. A., Stroup, W. W., Wolfinger, R. D., & Schabenberger, O. (2006). SAS for mixed models (2nd ed.). Cary, NC: SAS Institute Inc.

    Google Scholar 

  • Little, R. J. (1995). Modeling the drop-out mechanism in repeated-measures studies. Journal of the American Statistical Association, 90(431), 1112–1121.

    Article  Google Scholar 

  • Lorenzo-Luaces, L., DeRubeis, R. J., & Webb, C. A. (2014). Client characteristics as moderators of the relation between the therapeutic alliance and outcome in cognitive therapy for depression. Journal of Consulting and Clinical Psychology, 82(2), 368.

    Article  PubMed  PubMed Central  Google Scholar 

  • Lorenzo-Luaces, L., Driessen, E., DeRubeis, R. J., Van, H. L., Keefe, J. R., Hendriksen, M., & Dekker, J. (2017). Moderation of the alliance-outcome association by prior depressive episodes: Differential effects in cognitive-behavioral therapy and short-term psychodynamic supportive psychotherapy. Behavior Therapy, 48, 581–595.

    Article  PubMed  Google Scholar 

  • Maxwell, S. E., & Cole, D. A. (2007). Bias in cross-sectional analyses of longitudinal mediation. Psychological Methods, 12, 23–44. doi:10.1037/1082-989X.12.1.23.

    Article  PubMed  Google Scholar 

  • Maxwell, S. E., Cole, D. A., & Mitchell, M. A. (2011). Bias in cross-sectional analyses of longitudinal mediation: Partial and complete mediation under an autoregressive model. Multivariate Behavioral Research, 46(5), 816–841.

    Article  PubMed  Google Scholar 

  • Newman, M. G., Castonguay, L. G., Jacobson, N. C., & Moore, G. A. (2015). Adult attachment as a moderator of treatment outcome for generalized anxiety disorder: Comparison between cognitive–behavioral therapy (CBT) plus supportive listening and CBT plus interpersonal and emotional processing therapy. Journal of Consulting and Clinical Psychology, 83(5), 915–925. doi:10.1037/a0039359.

    Article  PubMed  PubMed Central  Google Scholar 

  • Padesky, C. A. (2004). Behavioral experiments: At the crossroads. In J. Bennett-Levy, G. Butler, M. Fennell, A. Hackmann, M. Mueller & D. Westbrook (Eds.), Oxford Guide to Behavioral Experiments in Cognitive Therapy (pp. 433–438). Cambridge: Oxford University Press.

    Chapter  Google Scholar 

  • Pavot, W., & Diener, E. (1993). Review of the satisfaction with life scale. Psychological Assessment, 5(2), 164–172.

    Article  Google Scholar 

  • Pierson, H., & Hayes, S. C. (2007). Using acceptance and commitment therapy to empower the therapeutic relationship. The therapeutic relationship in cognitive behavior therapy (pp. 205–228). London: Routledge.

    Google Scholar 

  • Rapaport, M. H., Clary, C., Fayyad, R., & Endicott, J. (2005). Quality-of-life impairment in depressive and anxiety disorders. American Journal of Psychiatry, 162, 1171–1178.

    Article  PubMed  Google Scholar 

  • Rubel, J. A., Rosenbaum, D., & Lutz, W. (2017). Patients’ in-session experiences and symptom change: Session-to-session effects on a within-and between-patient level. Behaviour Research and Therapy, 90, 58–66.

    Article  PubMed  Google Scholar 

  • Safran, J. D., & Muran, J. C. (2000). Negotiating the therapeutic alliance: A relational treatment guide. New York, NY: Guilford Press.

    Google Scholar 

  • Sasso, K. E., Strunk, D. R., Braun, J. D., DeRubeis, R. J., & Brotman, M. A. (2015). Identifying moderators of the adherence-outcome relation in cognitive therapy for depression. Journal of Consulting and Clinical Psychology, 83(5), 976–984. doi:10.1037/ccp0000045.

    Article  PubMed  PubMed Central  Google Scholar 

  • Sasso, K. E., Strunk, D. R., Braun, J. D., DeRubeis, R. J., & Brotman, M. A. (2016). A re-examination of process–outcome relations in cognitive therapy for depression: Disaggregating within-patient and between-patient effects. Psychotherapy Research, 26(4), 387–398.

    Article  PubMed  Google Scholar 

  • Strunk, D. R., Brotman, M. A., & DeRubeis, R. J. (2010). The process of change in cognitive therapy for depression: Predictors of early inter-session symptom gains. Behaviour Research and Therapy, 48(7), 599–606.

    Article  PubMed  PubMed Central  Google Scholar 

  • Tracey, T. J., & Kokotovic, A. M. (1989). Factor structure of the Working Alliance Inventory. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 1, 207–210. doi:10.1037/1040-3590.1.3.207.

    Article  Google Scholar 

  • Üstün, T. B., Ayuso-Mateos, J. L., Chatterji, S., Mathers, C., & Murray, C. J. (2004). Global burden of depressive disorders in the year 2000. The British Journal of Psychiatry, 184(5), 386–392.

    Article  PubMed  Google Scholar 

  • Wang, L. P., & Maxwell, S. E. (2015). On disaggregating between-person and within-person effects with longitudinal data using multilevel models. Psychological Methods, 20(1), 63.

    Article  PubMed  Google Scholar 

  • Weiss, M., Kivity, Y., & Huppert, J. D. (2014). How does the therapeutic alliance develop throughout cognitive behavioral therapy for panic disorder? Sawtooth patterns, sudden gains, and stabilization. Psychotherapy Research, 24(3), 407–418.

    Article  PubMed  Google Scholar 

  • Whisman, M. A. (1993). Mediators and moderators of change in cognitive therapy of depression. Psychological Bulletin, 114(2), 248.

    Article  CAS  PubMed  Google Scholar 

  • Wong, C. W. (2013). Collaborative empiricism in culturally sensitive cognitive behavior therapy. Cognitive and Behavioral Practice, 20(4), 390–398.

    Article  Google Scholar 

  • World Values Survey (2009). European and World values surveys five-wave integrated data file, 1981–2005. The European Values Study Foundation and World Values Survey Association, Retreived at http://www.worldvaluessurvey.org/.

  • Zilcha-Mano, S. (2016). New analytic strategies help answer the controversial question of whether alliance is therapeutic in itself. World Psychiatry, 15(1), 84–85.

    Article  PubMed  PubMed Central  Google Scholar 

  • Zilcha-Mano, S. (2017). Is alliance really therapeutic? A systematic answer based on recent methodological developments. American Psychologist, 72(4), 311–325.

    Article  PubMed  Google Scholar 

  • Zilcha-Mano, S., Dinger, U., McCarthy, K. S., & Barber, J. P. (2014). Does alliance predict symptoms throughout treatment, or is it the other way around? Journal of Consulting and Clinical Psychology, 82(6), 931.

    Article  PubMed  Google Scholar 

  • Zilcha-Mano, S., Dinger, U., McCarthy, K. S., Barrett, M. S., & Barber, J. P. (2014). Changes in well-being and quality of life in a randomized trial comparing dynamic psychotherapy and pharmacotherapy for major depressive disorder. Journal of Affective Disorders, 152, 538–542.

    Article  PubMed  Google Scholar 

  • Zilcha-Mano, S., & Errázuriz, P. (2015). One size does not fit all: Examining heterogeneity and identifying moderators of the alliance–outcome association. Journal of counseling psychology, 62(4), 579.

    Article  PubMed  Google Scholar 

  • Zilcha-Mano, S., & Errázuriz, P. (2017). Early development of mechanisms of change as a predictor of subsequent change and treatment outcome: The case of working alliance. Journal of Consulting and Clinical Psychology, 85(5), 508–520.

    Article  PubMed  Google Scholar 

  • Zilcha-Mano, S., Muran, J. C., Hungr, C., Eubanks, C. F., Safran, J. D., & Winston, A. (2016). The relationship between alliance and outcome: Analysis of a two-person perspective on alliance and session outcome. Journal of Consulting and Clinical Psychology, 84(6), 484. doi:10.1037/ccp0000058.

    Article  PubMed  PubMed Central  Google Scholar 

  • Zimmerman, M., McGlinchey, J. B., Young, D., & Chelminski, I. (2006). Diagnosing major depressive disorder I: A psychometric evaluation of the DSM-IV symptom criteria. The Journal of Nervous and Mental Disease, 194(3), 158–163.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sigal Zilcha-Mano.

Ethics declarations

Conflict of Interest

Sigal Zilcha-Mano, Ilana Lipsitz, and Paula Errázuriz, declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zilcha-Mano, S., Lipsitz, I. & Errázuriz, P. When is it Effective to Focus on the Alliance? Analysis of a Within-Client Moderator. Cogn Ther Res 42, 159–171 (2018). https://doi.org/10.1007/s10608-017-9867-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10608-017-9867-4

Keywords

Navigation