Using TEI Mark-up and Pragmatic Classification in British Telecom Correspondence Corpus
Construction and analysis of the British Telecom Correspondence Corpus involving TEI mark-up and pragmatic classification. The project explores the history and preservation of BT archives, focusing on the digitization and cataloging of documents, photographs, and correspondence for easier access and research purposes.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Using TEI mark-up and pragmatic classification in the construction and analysis of the British Telecom Correspondence Corpus. Ralph Morton, Coventry University mortonr2@uni.coventry.ac.uk Twitter @RalphMortonCov
Outline Project background British Telecom, New Connections, original aims, initial outcomes Working with Text Encoding Initiative (TEI) compliant XML in corpus mark-up Pragmatic Classification British Telecom Correspondence Corpus uses and future research
BT (British Telecom) The main telephone network in the United Kingdom between 1912 and 1984 (as a government department then public corporation). Traces its history back to founding of Electric Telegraph Company in 1846 In 1984 British Telecom was privatised One condition of the privatisation was the preservation of their public records
BT Archives Public archive Located in Holborn Established in 1986 Preserves the records of BT and its predecessors and promotes access to the records and their content internally as a corporate resource, and externally to national and international communities http://www.btplc.com/Thegroup/BTsHistory/BTgrouparchives/OurHeritagePolicy/BTA_policies_2010_06.pdf
New Connections Project JISC-funded collaboration between Coventry University, BT Heritage and The National Archives Project aim to catalogue, digitise and develop a searchable online archive of almost half a million photographs, images, documents and correspondence assembled by BT over 165 years.
Promoting easier access to the archives - not just two days a week, not limited to Holborn High Street. Engaging with material in new ways Three research projects attached to New Connections, one of which is the British Telecom Correspondence Corpus (BTCC)
British Telecom Correspondence Corpus Original aims - to identify and transcribe around 500 letters - collect contextual information for those letters and encode it using TEI compliant XML - Use corpus analyses to gain new insights into how English business correspondence changed from the mid-nineteenth to late- twentieth century
Corpus vs. Archive Leech (1991:11) - ultimately, the difference between an archive and a corpus must be that the latter is designed or required for a particular 'representative' function . Hunston (2002: 28) being representative inevitably involves knowing what the character of the whole is .
The character of the whole of the BT Archive is unknown at the item level In cataloguing, BT archivists describe the context and function of the folder in relation to the history of BT and its predecessors and not the individual authors of letters (Sian Wynn-Jones, personal communication 05/03/2013). archives are organised not classified (David Hay 2013, personal communication, 2013) We were provided with sufficient Category C files to fulfil our initial request for 500 letters. Letters were extracted through manual examination of digitised folders
One of the advantages of letters is they are rich with contextual data Took an approach normally reserved for spoken language which exists in unknowable quantities and in an unknowable range of varieties (Hunston, 2002: 29) Selected a number of factors to control and sampled accordingly: a purposive approach.
Sampling Balance between decades as far as possible, incl. every letter from underrepresented decades to be 'internally contrastive (Sinclair 2005) Variety of authors- Including historically interesting letters BUT inclusion of day to day letters too so as to not only preserve histories of prominent individuals (Nurmi, 1999:54) (Prescott 2012). Inclusion of handwritten and typed letters Where available, inclusion of chains of correspondence (Dossena 2004)
Basic metadata extraction Time constraints meant that the initial sampling metadata was very basic. (see right) Date Author Recipient General topic of the letter Whether the letter was part of a chain Format (handwritten etc)
Initial Findings - 612 letters (150 OCR, 462 manual transcription) - 386 authors - 132, 917 words 80 70 60 50 40 30 20 10 0 1850s 1860s 1870s 1880s 1890s 1900s 1910s 1920s 1930s 1940s 1950s 1960s 1970s 1980s
Detailed Metadata Begin to get an idea of the kind of document that the archive contains. Occupation Secretary most frequently listed profession (27) with a further 34 letters from variations on this role (assistant secretary, honourable secretary, under secretary, deputy secretary...). Need for caution when generalising about jobs, case of Secretary. 1934 -> Director General . 1966 -> Deputy Chairman of the Post Office Board . Area for investigation? We see a similar variety in the role of the next most common occupation Director, where there are 17 letters from Director and 13 letters from variations on this (e.g. deputy managing director).
Companies The correspondence originates from 137 separate companies. Aside from The Post Office/BT the letters predominantly come from communications companies and press organisations. There are also letters from government departments, law firms, charities, universities, district councils and miscellaneous one-off letters from organisations like the National Rifle Association and the Belgian Citizen Band Association
Gender 257 Male, 16 Female authors (106 where it s not clear from letter or outside sources) Surprising? [women] were eventually deemed capable of replacing men in labour intensive activities like sorting. But more intellectually demanding positions such as clerical posts in the Chief Engineer s Department remained firmly closed to them (Duncan Campbell-Smith, 2012:246)
Working with TEI The Text Encoding Initiative (TEI) is a consortium which collectively develops and maintains a standard for the representation of texts in digital form. Text internal <body> Header Information <header>
Correspondence SIG TEI Proposal Focus on header information <profileDesc> <correspDesc> <correspAction type="sending"> <persName>The sender</persName> <placeName>The place of sending</placeName> <date>The date of sending</date> </correspAction> <correspAction type="receiving"> <persName>The recipient</persName> <placeName>The place of receiving</placeName> <date>The date of receiving</date> </correspAction> </correspDesc> </profileDesc> Context?
CorrespDesc in the BTCC <ct:correspAction type="sending"> <persName> <forename>Henry</forename> <surname>Sch tz-Wilson</surname> <roleName>Assistant Secretary</roleName> </persName> <placeName> <orgName type="Company">The Electric & International Telegraph Company</orgName> <address> <street>Telegraph Street</street> <settlement>London</settlement> <postCode>EC</postCode> </address> </placeName> </ct:correspAction>
Fields recorded in our metadata Resource creation BT/Post Office file references maintaining links Transcription information <encodingDesc> - project description Letter metadata <persName> - <sender> and <addressee> <roleName> - occupation <orgName> - company, department <placeName> - <sender> and <addressee> - location info <date> yyyy-mm-dd, n= decade <keywords> - topic, function <sex> <scriptDesc> - format (handwritten, typed etc)
Text Internal <text> <body> <opener> <salute>Dear Sir,</salute> </opener> <p>In reply to your favor of yesterday I hasten to forward you the full particulars connected with the matter between us and the International Commissioners and Copy of the Correspondence which took place on the subject.</p> <closer> <salute>In haste, Yours faithfully</salute> <signed>L. Walter Courtenay</signed> </closer> </body> </text> Allows us to look at individual textual features of letter. E.G. use of <postScript> <quote>, <soCalled>, use of letter titles and references and letter <opener> and <closer>s including salutations.
<openers> 90% 80% 70% Sir 60% Dear Sir 50% Dear [first name] 40% Dear [Surname] 30% Dear [title] [surname] 20% 10% 0% 1850s 1860s 1870s 1880s 1890s 1900s 1910s 1920s 1930s 1940s 1950s 1960s 1970s 1980s
Most frequent <closers> by decade 50 45 40 35 30 Yours sincerely 25 Obedient Yours faithfully 20 15 10 5 0 1850s 1860s 1870s 1880s 1890s 1900s 1910s 1920s 1930s 1940s 1950s 1960s 1970s 1980s
Formal links between openings and closings 50 45 40 35 30 Named recipients 25 Yours sincerely 20 15 10 5 0 1850s 1860s 1870s 1880s 1890s 1900s 1910s 1920s 1930s 1940s 1950s 1960s 1970s 1980s
60 50 40 Unnamed recipients "polite" sign offs 30 20 10 0 1850s 1860s 1870s 1880s 1890s 1900s 1910s 1920s 1930s 1940s 1950s 1960s 1970s 1980s yours faithfully , obedient servant
Marconi Uses combination of relatively familiar - Dear Mr Preece In combination with a variety of familiar and formal closing salutations believe me dear sir yours very truly with best regards for you and all your family I remain dear sir yours very truly I remain dear sir yours very sincerely I remain dear sir yours very truly and sincerely
To be used as a starting point. There are many other factors to consider Context - relationship between sender and addressee - population of authors Content and function of letter need to be taken into account. Salutations may even provide clues to nature of correspondence, e.g. Nesfield s demi-official correspondence (1917:191) One way into the analysis
Letter function One of the challenges in approaching the analysis of the BT corpus is in how to make meaningful comparisons across so many different years, authors and subject matters. To try and address this we categorised the letters pragmatic by function. Looking at how these functions are realised and how they have changed/remained stable over time
Definitions were generated through a close examination of the letters. Primary Functions Advice, Suggestion, (Instruction?), Request, Application, Offer, Confirmation, Agreement, Acceptance, Rejection, Outlining, Detailing, Setting Out, Report, Notification, Expressive, Query, Clarification, Reiteration, Correction, Explanation, Complaints, Reminder, Thanking, Enclosing, Forwarding, Copying, Acknowledging, Arguing, Disputing, Arranging, Planning, Instructing, Personal Update, Proposal, Expenditure Review, Commissive, Promise Secondary Functions Thanking, Apology, Acknowledgement, Expressing, Query, Request, Offer, Advice, Suggestion, Direction, Instruction, Recommendation, Discussion, Informing, Stating, Agreeing, Conceding, Noting Change, Restating, Explanation, Invitation, Report, Notification, Enclosure, Approval Long list was narrowed down to a more manageable list of 19 functions This list was tested at a workshop at Coventry University with six participants asked to identify the main function (+component functions) of a sample of letters
Frequent problems Lack of context What s the main function? E.g. complaints Form/function conflict - Overlap offer vs. application
Problem cases author identified functions Dear Sir Robert: <informative>When you spoke to me on the telephone on July 18th, I had already received a telegram from Pattison summarizing his conversation with you a few days ago . particularly in the matter of possible reference to technical detail, and that the objectives to be stressed should relate to a greater degree to exploration of general policy considerations.</informative> <thanking>However, I need not amplify on these matters in this letter since we are doing our best to get the formal reply to the British despatch out as soon as we can. I am sending this personal note to you, however, to express my appreciation for your own letter ..
Problem cases - multifunctionality. There are some examples of letters where the authors address multiple independent concerns which carry equal claim to being the primary function of the letter. E.g. 1917_05_19_WM_## reproduced in full below Sir, <informative> I am directed to refer to your communication dated 16th May, 1917, No. 96951/17 relating to Mr.T.Gilbert, a Telegraph Operator employed at the Ware Post Office and to inform you that instructions have this day been given for this man to be posted to the Royal Engineers, Signal Service, Bletchley through Area Headquarters. </informative> <directive> I am also to ask you to forward to this Department for countersignature on behalf of the Director of Recruiting, your stock of enlistment Forms 27/Gen.No./6112 (D.R.l.c.) a copy of which was attached by you. It is hoped that this will obviate any further difficulty of the nature referred to in your communication. </directive> I am, Sir, Your obedient Servant, W. MacDonald The informative part of the letter does not contextualise the directive; the two sections address independent concerns.
Three more rounds of inter-rater testing, one with two participants, and two further rounds with three participants Improved pre-discussion agreement c. 80% two raters and 60% three raters -> 82% two raters and 70% three raters Clarifications
Final categories 1. Application, 2. Commissive, 3. Complaint, 4. Declination, 5. Directive, 6. Informative 7. Notification, 8. Offer, 9. Query, 10. Thanking
Application Commissive Complaint Declination Directive InformativeNotification Offer Query Thanking 1850s 0 0 0 0 1 2 1 3 0 0 1860s 0 0 1 0 1 1 5 0 2 0 1870s 3 5 1 2 9 14 0 3 3 0 1880s 0 4 1 8 2 1 0 0 3 0 1890s 0 2 2 1 8 16 5 2 5 1 1900s 1 10 3 3 4 6 2 3 2 0 1910s 0 2 0 8 11 11 6 3 3 0 1920s 10 5 2 3 14 14 6 8 4 1 1930s 0 3 4 7 7 11 3 5 4 3 1940s 2 6 3 2 10 18 1 3 4 0 1950s 2 2 4 2 10 13 5 2 4 3 1960s 0 11 0 0 3 21 7 5 1 3 1970s 0 1 4 2 8 14 5 4 2 0 1980s 0 1 3 3 1 20 4 2 2 0 Total 18 52 28 41 89 162 50 43 39 11
Analysis Data Driven analysis of the <text> <body> Using Frequency Lists, Keywords and Clusters as starting points - By decade (diachronic) - By function
Application 1 19 2 25 3 35 4 14 5 7 6 10 7 5 8 13 9 10 10 10 11 4 12 5 13 6 14 5 15 9 47.908 york 30.214 new 25.294 my 22.262 call 20.102 times 19.432 application 18.747 salary 18.447 experiments 18.035 beg 16.529 years 16.410 grove 16.368 opening 15.793 hoping 15.704 convenience 15.044 request
Directive 1 2 3 4 5 6 7 8 9 10 11 12 13 14 80 15 148 12 11 20 444 14 12 12 11 31 126 9 33.780 committee 19.098 landlord 17.766 if 15.278 tenant 14.005 bbc 12.847 calls 12.836 be 12.619 ireland 12.477 rayner 12.477 television 12.082 northern 11.971 majesty 11.924 should 11.459 ita
Far from perfect inter-rater reliability Issues with multi-functional letters, and component functions Not much data so individual letters can skew results BUT Some promising preliminary results. Need to examine patterns in corpus as a whole and back any claims up with close reading Could be supplemented by qualitative approaches such as analysis of rhetorical moves (see Biber et al. 2007)
BTCC: Current use and future directions Who uses it? - (currently) me - Corpus will be available by request/through e.g. Oxford Text Archive - BT Archives have the data and are looking to incorporate it in their Digital Archive
For What? Linguistic analysis - Somewhat exploratory. Cannot generalise but planning to expand corpus to examine findings further Historical study - transcriptions, metadata
Ways in which historical research is being transformed by Digital Methods opportunity to read historical documents in new ways making publicly available but practically restricted material available providing historical archives with transcriptions and detailed item level metadata potentially bringing related but separate physical archives material together in the form of a digital resource (Post Office)
Thank you! mortonr2@uni.coventry.ac.uk References Biber, Douglas, Connor, Ulla, and Upton, Thomas A., eds. Discourse on the Move : Using Corpus Analysis to Describe Discourse Structure. Amsterdam, NLD: John Benjamins Publishing Company, Dossena, M. (2004) Towards a corpus of nineteenth-century Scottish correspondence Linguistica e Filologia 18, 195-214 Hunston, S. (2002) Corpora in Applied Linguistics Cambridge, Cambridge University Press Leech, G. (1991) The state of the art in corpus linguistics In Aijmer, K. & Altenberg, B. (eds.) English Corpus Linguistics: Studies in honour of Jan Svartvik, 8- 29 Nesfield, J.C. (1917) Junior Course of English Composition, London, MacMillan and Co. Ltd Prescott, A (2012) Making the Digital Human: Anxieties, Possiblities, Challenges delivered at Digital Humanities Summer School, Merton College Oxford [online] available from http://digitalriffs.blogspot.co.uk/2012/07/making-digital-human- anxieties.html [20th March 2013]
