Case study: Job Satisfaction in the BankingIndustry: Or the logistical nightmare of conducting large scalequantitative research
Jobcharacteristics are believed to have an impact on stress andwell-being at work (Karasek & Theorell, 1990). The demands of the job onthe one hand and the extent towhich you have control over your own activities (decision latitude)on the other, are two factors which together define how stressful ajob is. Those jobs which are high demand, but offer limitedcontrol, are considered to be high-strain and carry an increasedrisk of job dissatisfaction, stress and burnout.
Based on thistheoretical framework, the Union of Belgian Banks sent out aresearch call to several institutions, with a bidding process basedon criteria such as quality of the proposal, timing, and â aboveall â budget. The aim of the research was to carry out quantitativeresearch to measure the relationship between job characteristicsand job satisfaction in all Belgian banks (see Cambré etal., forthcoming). But in order to do this effectively, severalmethodological issues needed to be resolved during the researchprocess.
First of all,a research consortium was selected to conduct the research, or moreprecisely, the two highest ranked bidders were asked to jointlyundertake the research. This was the outcome of a politicaldecision by the banks (see also p. 142, âAffiliation and conflictsof interestâ),since the employers preferred onepartner and the unions (employee representatives) preferred theother. The two competing research institutes, a private companyspecialising in stress at work and the KatholiekeUniversiteit Leuven (Belgium), were required to co-operate anddevelop a level of trust in order to conduct the research. Forexample, both research institutes had different ideas as to whichscale should be used in the questionnaire. They could not justcombine the scales or include both scales, simply because they aresupposed to measure the same concept. Furthermore, this would alsomake the questionnaire too complex. Therefore, the researchinstitutes had to combine their knowledge, look for compromises andjointly work on a shared vision, which is, to say the least, rathertime consuming.
A second obstaclethat needed to be overcome was the sample(see Chapter 7). Intotal, 69,000 employees work forBelgian banks and it was decided that questioning all employeeswould be too complicated and too expensive. Therefore the researchcommittee, consisting of representatives of the banks, the unionsand the research consortium, opted for a cross-sectional design (p.45) with a fixed sample of 15,000 employees (roughly 21%; see p.187 âAbsolute and relative sample sizeâ).
In this sample,the small banks were over-represented in order to be able to makeconclusions at the level of each bank. Within each bank, therespondents were selected at random with no particular quota forgender, age or employee level. In the postal survey (see p.231âSelf-completion questionnaire or postal questionnaireâ) severalsteps were taken to improve the response rate (see p. 234 âSteps toimprove response rates to postal questionnaireâ; see alsosuggestions by Dillman,1983). The survey was based on addresses which had been provided bythe banks (name, language, address) and each employee randomlyselected in the sample received a personalized envelope throughregular mail, sent to him/her by the employer. The completedquestionnaire needed to be returned (free of charge) through theinternal post within each bank. This caused two problems: (1) aperceived lack of anonymity, because the employees received apersonalized envelope (p.136, âInvasion of privacyâ); and (2)potential bias to the reliability (p.41; p.157) and the responserate (see Key concept 7.5 on p. 189) because the completedquestionnaires were collected by the banks themselves. Theresearchers were able to overcome this by communicating clearly ina letter (1) that although the data collection was not completelyanonymous (home address on envelope), the data analysis would becompletely anonymous; and (2) that the completed questionnaireswere collected by the bank but were transferred immediately to theresearchers without being opened or read.
The latteralready presented various logistical problems. The researchers had to travel to eachbank to collect the completed questionnaires and due to the factthat in Belgium, part of the population speak Flemish (Dutch) andpart speak French, two versions of the questionnaire needed to beavailable and then carefully translated and tested for the accuracyof their translation (see also Tips and skills, p.488: âTranslatinginterview dataâ). The questionnaires were sent to the respondentsâhome addresses, a French version if the respondent lived in theFrench part of Belgium, a Flemish version when living in Flanders.This prompted a series of angry calls when Flemish people, livingin the French part, or vice versa, received a questionnaire thatwas not in their native language. Furthermore, Brussels isofficially bi-lingual and, to complicate matters even more,contains many headquarters in which the main language spoken is ...English! In order to minimise attrition, it was important thatthese respondents received a questionnaire in their preferredlanguage. Another logistical issue was the co-ordination andcontrol of the distributed information. The Belgian banks, who werethe research financers, chose a decentralised way of working, henceorganizing a âsensibilization campaignâ within each bank whereby theresearchers had to visit all the banks to explain the theoreticalframework and the outline of the research to representatives ofboth employers and employees.
Additional initiativesto prompt a higher response rate were taken up by individual banks,or, more precisely, by some of the banks. The researchers wererequired to carefully follow-up on those initiatives implemented bythe banks, to ensure that these initiatives remained both neutraland valid for the research. Some of these initiatives proveddifficult to deal with due to the selective use of information thathad been employed (e.g. letters forcing the employees toparticipate; or union campaigns to guide certain answers). Hence,the researchers had to be sensitive for the respectiveorganisational cultures, while making sure they kept a neutralposition towards all partners involved in the research.
Once the datacollection was completed (response rate of 47.6%), thedatahandling needed muchattention. A comprehensive check and double-check was conducted onwrong entries, filters, missing cells ... just to increase thereliability as well as calculating a weight-factor to compensatefor the over-representation of small banks. One issue was the majordifference in response rate between the banks. Due to a strongcampaign, some banks reached a response rate of over 60%, whilstothers barely reached 20% because they did nothing to increase theresponse rate. To what extent did this bias the reliability? At therequest of the financers, a reference group of Belgian employeesneeded to be defined, comparable on gender, age, educational leveland employee level which would allow all banks to be compared notonly with the other banks, but also against an overall group ofemployees outside of the banking sector. However, since thequestionnaire was also the result of a negotiation, some of theitems used were new and not included in previous research oneconomy-wide employee well-being. Obviously, there existed noprevious data for these new items. As a consequence, these newitems could not be compared with larger populations and thereforeonly an internal comparison (every bank compared to all otherbanks) was chosen. A final issue here was the choice for a minimalthreshold for analysis and reporting. It was decided that everycell needed to contain at least 15 observations in order to allowfor a graphical representation in the analysis or the final report.As a result, for some smaller banks the final report was notexactly fine-grained, since some of these banks only contain 25employees. For these banks, only overall results were presented,whereas for larger banks the results were split up for gender, age,department, etc.
A final issueoccurred when presenting the results. As mentioned above, the languageissue is particularly important in Belgium to the extent that oneeven has to be concerned with the order of reporting and presenting(in terms of which language first).
A discussion aroseconcerning the graphs used in the report: using different axes canresult in different perspectives, despite the fact that,statistically, the results obviously remain the same. In bothfigures below, the amount of people with stress is 5, whereas 10have no stress. So the appearances can be deceptive.
questions â
1)⢠Sample size is n=15,000. Is this large sample reallynecessary? Discuss its relative and absolute size. What otheroptions could have been taken?
2)⢠This is research in a real socialcontext. Hence issues such astime, budget and politics are important. How would you deal withthem? Discuss some of the decisions the researchers havemade.
3)⢠The small banks were over-represented in the sample. Afterwards, a weight factor wascalculated in order to correct for this. However, as could beexpected, for some small banks the response rate was still toosmall to allow for organizational conclusions. So what was thepoint of this over-representation? Discuss its appropriateness incontemporary organizational research with many small firms. Whatalternatives could have been used to allow for conclusions atorganizational level?
4)⢠Due to a strong campaign, some banks reacheda response rate of over 60%, while others hardly reached 20%. Towhat extent did this bias the reliability of the results? What canbe said about generalisation issue?
