В содержательном подходе количество информации, заключенное в сообщении, определяется объемом знаний, который это сообщение несет получающему его человеку.

Вспомним, что с «человеческой» точки зрения информация — это знания, которые мы получаем из внешнего мира. Количество информации, заключенное в сообщении, должно быть тем больше, чем больше оно пополняет наши знания.

Вы уже знаете, что за единицу измерения информации принимается \(1\) бит.

1 бит — минимальная единица измерения количества информации.

Сообщение, уменьшающее неопределенность знания в два раза, несет 1 бит информации.

Что такое неопределенность знания, поясним на примерах.

Допустим, вы бросаете монету, загадывая, что выпадет: орел или решка. Есть всего два возможных результата бросания монеты. Причем ни один из этих результатов не имеет преимущества перед другим. В таком случае говорят, что они равновероятны.

В случае с монетой перед ее подбрасыванием неопределенность знания о результате равна двум.

Игральный же кубик с шестью гранями может с равной вероятностью упасть на любую из них. Значит, неопределенность знания о результате бросания кубика равна шести.

Еще пример: спортсмены-лыжники перед забегом путем жеребьевки определяют свои порядковые номера на старте. Допустим, что имеется \(100\) участников соревнований, тогда неопределенность знания спортсмена о своем номере до жеребьевки равна \(100\).

Следовательно, можно сказать так:

Неопределенность знания о результате некоторого события (бросание монеты или игрального кубика, вытаскивание жребия и др.) — это количество возможных результатов.

Сообщение об одном из двух равновероятных результатов некоторого события несет 1 бит информации.

Тогда количество информации \(i\), содержащееся в сообщении о том, что произошло одно из \(N\) равновероятных событий, можно определить из формулы Хартли:

N=2i.

Данная формула является показательным уравнением относительно неизвестного \(i\).

i=log2N — логарифм \(N\) по основанию \(2\).

Если \(N\) равно целой степени двойки (\(2, 4, 8, 16\) и т. д.), то такое уравнение можно решить «в уме».

Пример:

Шахматная доска состоит из \(64\) полей: \(8\) столбцов на \(8\) строк.

Какое количество бит несет сообщение о выборе одного шахматного поля?

Решение.

Поскольку выбор любой из \(64\) клеток равновероятен, то количество бит находится из формулы:

2i=64,

i=log264=6, так как 26=64.

Следовательно,  \(i = 6\) бит.

В противном случае количество информации становится нецелой величиной, и для решения задачи придется воспользоваться таблицей двоичных логарифмов.

Также, если \(N\) не является целой степенью \(2\), то можно выполнить округление \(i\) в большую сторону. При решении задач в таком случае \(i\) можно найти как log2K, где \(K\) — ближайшая к \(N\) степень двойки, такая, что \(K > N\).

Пример:

При игре в кости используется кубик с шестью гранями.

Сколько битов информации получает игрок при каждом бросании кубика?

Решение.

Выпадение каждой грани кубика равновероятно. Поэтому количество информации от одного результата бросания находится из уравнения:2i=6.

Решение этого уравнения: i=log26.

Из таблицы двоичных логарифмов следует (с точностью до \(3\)-х знаков после запятой):
\(i = 2,585\) бита.

Данную задачу также можно решить округлением \(i\) в большую сторону:  2i=6<8=23,i=3 бита.

Источники:

Семакин И. Г. Информатика и ИКТ. Базовый уровень : учебник для 10-11 классов / И. Г. Семакин, Е. К. Хеннер. — 8-е изд. — М. : БИНОМ. Лаборатория знаний, 2012, стр. 21-24

Информатика и ИКТ. Задачник-практикум : в 2т. Т. 1 / Л. А. Залогова [и др.] ; под ред. И. Г. Семакина, Е. К. Хеннера. — 3-е изд. — М. : БИНОМ. Лаборатория знаний, 2011, стр. 15-16

A Task-based approach | TeachingEnglish | British Council

This article also links to the following activity.
Try — Speaking activities — Task-based speaking — planning a night out

Present Practice Produce (PPP)
During an initial teacher training course, most teachers become familiar with the PPP paradigm. A PPP lesson would proceed in the following manner.

• First, the teacher presents an item of language in a clear context to get across its meaning.This could be done in a variety of ways: through a text, a situation build, a dialogue etc.

• Students are then asked to complete a controlled practice stage , where they may have to repeat target items through choral and individual drilling, fill gaps or match halves of sentences. All of this practice demands that the student uses the language correctly and helps them to become more comfortable with it.

• Finally, they move on to the production stage, sometimes called the ‘free practice’ stage.Students are given a communication task such as a role play and are expected to produce the target language and use any other language that has already been learnt and is suitable for completing it.

The problems with PPP
It all sounds quite logical but teachers who use this method will soon identify problems with it:

• Students can give the impression that they are comfortable with the new language as they are producing it accurately in the class.Often though a few lessons later, students will either not be able to produce the language correctly or even will not produce it at all.

• Students will often produce the language but overuse the target structure so that it sounds completely unnatural.

• Students may not produce the target language during the free practice stage because they find they are able to use existing language resources to complete the task.

A Task-based approach
Task -based learning offers an alternative for language teachers.In a task-based lesson the teacher does not pre-determine what language will be studied, the lesson is based around the completion of a central task and the language studied is determined by what happens as the students complete it. The lesson follows certain stages.

Pre-task
The teacher introduces the topic and gives the students clear instructions on what they will have to do at the task stage and might help the students to recall some language that may be useful for the task.The pre-task stage can also often include playing a recording of people doing the task. This gives the students a clear model of what will be expected of them. The students can take notes and spend time preparing for the task.

Task
The students complete a task in pairs or groups using the language resources that they have as the teacher monitors and offers encouragement.

Planning
Students prepare a short oral or written report to tell the class what happened during their task.They then practise what they are going to say in their groups. Meanwhile the teacher is available for the students to ask for advice to clear up any language questions they may have.

Report
Students then report back to the class orally or read the written report. The teacher chooses the order of when students will present their reports and may give the students some quick feedback on the content. At this stage the teacher may also play a recording of others doing the same task for the students to compare.

Analysis
The teacher then highlights relevant parts from the text of the recording for the students to analyse. They may ask students to notice interesting features within this text. The teacher can also highlight the language that the students used during the report phase for analysis.

Practice
Finally, the teacher selects language areas to practise based upon the needs of the students and what emerged from the task and report phases. The students then do practice activities to increase their confidence and make a note of useful language.

The advantages of TBL
Task-based learning has some clear advantages

• Unlike a PPP approach, the students are free of language control. In all three stages they must use all their language resources rather than just practising one pre-selected item.

• A natural context is developed from the students ‘experiences with the language that is personalised and relevant to them. With PPP it is necessary to create contexts in which to present the language and sometimes they can be very unnatural.

• The students will have a much more varied exposure to language with TBL. They will be exposed to a whole range of lexical phrases, collocations and patterns as well as language forms.

• The language explored arises from the students ‘needs. This need dictates what will be covered in the lesson rather than a decision made by the teacher or the coursebook.

• It is a strong communicative approach where students spend a lot of time communicating.PPP lessons seem very teacher-centred by comparison. Just watch how much time the students spend communicating during a task-based lesson.

• It is enjoyable and motivating.

Conclusion
PPP offers a very simplified approach to language learning. It is based upon the idea that you can present language in neat little blocks, adding from one lesson to the next. However, research shows us that we can not predict or guarantee what the students will learn and that ultimately a wide exposure to language is the best way of ensuring that students will acquire it effectively.Restricting their experience to single pieces of target language is unnatural.

For more information see ‘A Framework for Task-Based Learning’ by Jane Wills, Longman; ‘Doing Task-Based Teaching’ by Dave and Jane Willis, OUP 2007.
Also see www.willis-elt.co.uk

Richard Frost, British Council, Turkey

Task-based Learning — TBL Lesson

Task-based Learning (TBL)

Task-based learning (TBL) is a teaching method that focuses on context and meaning. This approach is also called task-based instruction (TBI) or task-based language teaching (TBLT).

In a task-based language learning class, teachers give students tasks to compete so that they can practise the language in a personalised and meaningful way.

After completing the task, the teacher asks students to consider the language they used.However, the main focus of a task-based learning class is on the students actually doing the task itself. This reveals the language that is studied.

Task-based language learning uses practical tasks to help students find their own useful vocabulary and language structures.

Example tasks might be going food shopping, visiting the doctor, dealing with issues in an airport, making a telephone call, being interviewed or conducting an interview, ordering a taxi, complaining about a meal in a restaurant or resolving an issue in a shop .Task-based learning allows students to uncover their own vocabulary during the task.

Task-based learning vs PPP technique

There is much discussion over the best way to teach a language lesson. The PPP method of teaching is often used in the classroom as a way of introducing vocabulary or ideas.

The focus in PPP classes is on presentation, practice and production. However some people think that the PPP technique is old fashioned. Some teachers offer task-based learning as a viable alternative to PPP as a more practical teaching method.

A problem with PPP is that students can sometimes end up using unnatural ways to practice new language structures. Task-based learning is supposed to overcome that problem by putting all the emphasis on useful and meaningful tasks as the way to actually learn the language and vocabulary.

Image source

Students ‘role in task-based language learning

In a task-based language lesson, the teacher does not decide beforehand what language will be taught and learned.

The teacher prepares a task for the students to complete and the language and vocabulary learned is decided upon and produced naturally during the task. This way, the vocabulary learned will be student-led vocabulary that is always meaningful and useful.

A task-based learning or TBL lesson allows the students to take a more central role in determining the language structures they learn. The completion of the main task requires students to think for themselves about what they need to learn and the learning process happens during the completion of the task.

Although teachers do not pre-prepare the vocabulary as much as with a PPP lesson, a task-based language learning lesson does follow a pattern:

Task-based Language Learning (TBL) Lesson Structure

Pre-task preparation

Before the task begins, the teacher needs to present the topic and give instructions on the task. The teacher can also introduce some vocabulary that will be useful to students while they complete the task.

The preparation stage for a task-based language learning lesson might involve giving an example of the task being performed or even showing a video to students so they know what is expected of them.

The emphasis here should be on clarity of explanation, so student know exactly what to do during the task-based learning activity.

Image source

The Task

When students come to complete the task in a task-based learning (TBL) lesson, they can work individually, in pairs or in groups.

Normally it is best for students to work in small groups or pairs so they can use the language and practise together verbally and collaboratively.

The teacher should monitor the students ‘progress and offer encouragement or help where needed.

Planning and Reporting

The planning and reporting stage of a task-based learning lesson allows students to report back to the group, telling everyone else how they competed the task.

This can be the preparation of a verbal report or a written report. It can be formal in style or more informal in style and they can practice beforehand what they will say or write in their pairs.

This stage of the TBL lesson allows the students and the rest of the class to see what language each group needed and lets them practice their language in a natural way.

The teacher can also show videos of other groups performing a similar task to see how they competed it.

Image source

Analysis

During the analysis stage of the task-based learning class, the teacher can take elements from each of the students ‘reports and highlight the important language and any overlaps or interesting parts.

Students can discuss specific features of the task, how they approached them and what language skills were needed. If any video was shown during the reporting stage it is also a good idea to discuss how the students in the video completed their task.

Practice

Based on the reports and the analysis stage of the TBL lesson, the teacher can select important areas of the language that need extended practice.

The task and reports will show the areas where students had the most problems and therefore which language areas, words and phrases need practice.

Students can then do some activities to improve their knowledge and confidence with these problematic language areas.

Advantages of Task-Based Learning

A task-based language lesson is much more student-led than many other types of language lesson.In a task-based learning class, the students are more in control of what they learn. In all the stages of the class, the students can use their language skills in a meaningful way.

In many ways this makes a TBL lesson more natural that a PPP lesson, as the language is personalised and connected specifically to the context of the task.

The language explored and learned in a TBL class comes from the needs of the students. The needs uncovered during the task dictate what is learned, instead of this being decided beforehand by the teacher or a textbook.

Task-based learning is a strongly communicative approach to language teaching. In contrast, a PPP lesson is more teacher-driven.

Students need to communicate with each other in order to compete a task so they are forced to find new ways of using language and it quickly becomes clear what areas they find difficult and what areas of knowledge they are lacking in.

The tasks themselves make the TBL lesson motivating as students are engaged straight away and the language feels relevant.

Instead of restricting language lessons to focusing on a single piece of vocabulary or a single grammar point as is often the case in a PPP lesson, task-based language learning lessons let student uncover these areas naturally within a stimulating context.

Image source

Disadvantages of Task-Based Learning

Task-based learning has very little focus on accuracy. Because of this, students may find themselves practicing erroneous language a lot of the time.

The language required to complete a task and discuss it may be far above the level of many of the students. This could make the TBL class de-motivating for some students, particularly those of a lower ability level.

The way the groups complete the tasks is their choice. For this reason it is hard for the teacher to know exactly what language areas to introduce before the task.

Presenting certain vocabulary or language structures beforehand might be unnecessary if students do not require that language during the task.However, students could also feel frustrated if they find they do not already have the knowledge of the language points required during the task.

Although in theory, learning via a task can be more stimulating than an ordinary teacher-driven lesson, the motivation of the students depends on the task. The personalisation of the task-based learning (TBL) class can be negated by the task if it does not appeal to the individual student.

Indeed, if students use language they already know to compete the task during the TBL lesson, they are not actually learning anything new — they are only practicing.

Another problem in language learning via a TBL lesson is that task-based learning suits proactive students and confident collaborators. For students who are quiet, shy or simply more reflective in their approach to learning, this might not be the best way for them to learn or even practice a language.

Image source

Share your thoughts on task-based learning

Do you like the task-based language learning approach?

Is task-based learning more effective than PPP for language lessons?

Can you think of any more advantages of task-based learning?

How would you address the disadvantages of the TBL approach?

Page Not Found — The Sweet Setup

Guides & Courses

NEW: Simple Habits

Use routines to make your life a little bit more manageable, a little bit easier, and a whole lot better. »

Timeblocking

A simple and effective way to manage your tasks and your time. »

Manage and Develop Your Ideas

Improve your creative process, and save time, with a simple system for developing your ideas.Plus in-depth screencasts for MindNode. »

All the Things

Productivity and task management training, plus in-depth video tutorials for Things 3. »

Day One In Depth

The most extensive video guide to Day One available today, plus ideas and workflow examples. »

Learn Ulysses

Organize your writing and capture your ideas with this video course to help you quickly learn Ulysses.»

Mobile Photography

Extensive video tutorials and training to take your photos from average to awesome. »

Bundle up and Save

Save 35% ($ 145 discount) when you get all nine in-depth courses to complete your library. »

Our Latest Posts

The Productive, Focused Writer (A TSS Webinar)

This coming Wednesday, July 15, we are hosting our next TSS Webinar.This one is all about being a more productive and focus writer. The are a handful of common challenges that face just about anyone who does any sort of writing or note taking. But the good news is that they are surprisingly easy […] »

Using NotePlan 2 as a Digital Bullet Journal

NotePlan 2 combines your tasks and your calendar, so if you love time blocking like I do, you’re going to be excited about this application. It does all this in a system that any Bullet Journaller will recognize.»

5 Text Analytics Approaches: A Comprehensive Review

Are you receiving more feedback than you could ever read, let alone summarize? Maybe you’ve used Text Analytics methods to analyze free-form textual feedback?

These methods range from simple techniques like word matching in Excel to neural networks trained on millions of data points.

Here is my summary to break down these methods into 5 key approaches that are commonly used today.

What is Text Analytics?

Text analytics is the process of extracting meaning out of text.For example, this can be analyzing text written by customers in a customer survey, with the focus on finding common themes and trends. The idea is to be able to examine the customer feedback to inform the business on taking strategic action, in order to improve customer experience.

What is Text Analytic software?

To make text analytics the most efficient, organisations can use text analytics software, leveraging machine learning and natural language processing algorithms to find meaning in enormous amounts of text.

How is Text Analytics used by companies?

To take Thematic as an example, we analyze the free-text feedback submitted in customer feedback forms, which was previously difficult to analyze, as companies spend time and resource struggling to do this manually.

Subsequently, we use text analytics to help companies find hidden customer insights and be able to easily answer questions about their existing customer data. In addition, with the help of text analytics software such as Thematic, companies can find recurrent and emerging themes, tracking trends and issues, and create visual reports for managers to track whether they are closing the loop with the end customer.

Some Text Analytics background …

For a long time, I’ve been planning to write a post to clarify what’s possible in text analytics today, in 2018.

Throughout my career, I’ve spoken with many who are living through the pain of analyzing text and trying to find a solution.

Some try to reinvent the wheel by writing their own algorithms from scratch, others believe that Google and IBM APIs are the saviours, others again are stuck with technologies from the late 90’s that vendors pitch as «advanced Text Analytics».

I’ve spent the last 15 years in Natural Language Processing, specifically in the area of ​​making sense of text using algorithms: researching, creating, applying and selling the technology behind it.

My academic research resulted in algorithms used by hundreds of organizations (I’m the author of KEA and Maui). The highlight of my text analytics career was at Google, where I wrote an algorithm that can analyse text in languages ​​I do not speak.

And for the past 3 years, in my role as the CEO of Thematic I’ve learned a lot about what’s available in the market.

So, it’s fair to say, I’m qualified to speak on this topic.

I’ll try to be objective in my review, but of course, I’m biased because of my position. Happy to discuss this with anyone who is interested in providing feedback.

5 Text Analytics Methods and Examples

Here is my summary to break down these methods into 5 key approaches that are commonly used today.

Text Analytics Approach 1: Word Spotting

Let’s start with word spotting .First off, it’s not a thing!

The academic Natural Language Processing community does not register such an approach, and rightly so. In fact, in the academic world, word spotting refers to handwriting recognition (spotting which word a person, a doctor perhaps, has written).

There is also keyword spotting, which focuses on speech processing.

But to my knowledge, word spotting is not a used for any type of text analysis .

But I’ve heard frequently enough about it in meetings to include in this review.It’s loved by DIY analysts and Excel wizards and is a popular approach among many customer insights professionals.

The main idea behind text word spotting is this: If a word appears in text, we can assume that this piece of text is «about» that particular word. For example, if words like «price» or «cost» are mentioned in a review, this means that this review is about «Price».

The beauty of the word spotting approach is its simplicity.

You can implement word spotting in an Excel spreadsheet in less than 10 minutes.

Or, you could write a script in Python or R. Here ‘s how.

How to build a Text Analytics solution in 10 minutes

You can type in a formula, like this one, in Excel to categorize comments into «Billing», «Pricing» and «Ease of use»:

And voilà!

Here it is applied to a Net Promoter Score survey where column B contains open-ended answers to questions «Why did you give us this score»:

It probably took me less than 10 minutes to create this, and the result is so encouraging! But wait …

Everyone loves simplicity.But in this case, simplicity sucks

Various issues can easily crop up with this approach.

Here, I’ve annotated them for you.

Out of 7 comments, here only 3 were categorized correctly. «Billing» is actually about «Price», and three other comments missed additional themes. Would you bet your customer insights on something that’s at best 50 accurate?

When word spotting is OK

You can imagine that the formula above can be tweaked further.And indeed, I’ve talked to companies who hand-crafted massive custom spreadsheets and are very happy with the results.

If you have a dataset with a couple of hundred responses that you only need to analyze once or twice, you can use this approach. If the dataset is small, you can review the results and ensure high accuracy very quickly.

When word spotting fails

As for the downside? Please do not use word spotting:

• If you have any substantial amount of data, more than several hundred responses
• If you will not have time to review and correct the accuracy of each piece of text
• If you need to visualize the results (Excel will hear you swearing)
• If you need to share the results with your colleagues
• If you need to maintain the data consistently over time

There are also many other disadvantages to DIY word spotting, that we’ll discuss in the next post.I’ll also talk about what actually does work and is a good approach.

If you wish to build your own Text Analytics solution, check out our in-depth guide: How to build your own feedback analysis solution.

Text Analytics Approach 2. Manual Rules

The Manual Rules approach is closely related to word spotting. Both approaches operate on the same principle of creating a match pattern, but these patterns can also get quite complex.

For example, a manual rule could involve the use of regular expressions — something you can not easily implement in Excel.Here is a rule for assigning the category «Staff Knowledge» from a popular enterprise solution Medallia:

Majority of Text Analytics providers as well as many other smaller players, who sell Text Analytics as an add-on to their main offering, provide an interface that makes it easy to create and manage such rules. They also sometimes offer professional services to help with the creation of these rules.

The best thing about Manual Rules is that they can be understood by a person.They are explainable, and therefore can be tweaked and adjusted when needed.

But the bottom line is that creating these rules takes a lot of effort. You also need to ensure that they are accurate and maintain them over time.

To get you started, some companies come with pre-packaged rules, already organized into a taxonomy. For example, they would have a category «Price», with hundreds of words and phrases already pre-set, and underneath they might have sub-categories such as «Cheap» and «Expensive».

They may also have specific categories setup for certain industries, e.g. banks. And if you are a bank, you just need to add your product names into this taxonomy, and you’re good to go.

The benefit of this approach is that once set up, you can run millions of feedback pieces and get a good overview of the core categories mentioned in the text.

But, there are plenty of disadvantages for this approach, and in fact any manual rules and word spotting technique:

1.Multiple word meanings make it hard to create rules

The most common reason why rules fail stems from polysemy , when the same word can have different meanings:

2. Mentioned word! = Core topic

Just because a word or a phrase is mentioned in text, it does not always mean that the text is about that topic. For example, when a customer is explaining the situation that leads to an issue: « My credit card got declined and the cashier was super helpful, waiting patiently while I searched for cash in my bag .»This comment is not about credit cards or cash, it’s about the behavior of the staff.

3. Rules can not capture sentiment

Knowing the general category alone is not enough. How do people think about «Price», are they happy or not? Capturing sentiment with manually pre-set rules is impossible. People often do not realize how diverse and varied our language is.

So, a sub-category like «expensive» is actually extremely difficult to model.A person could say something like « I did not think this product was expensive «. To categorize this comment into a category like «good price», you would need a complex algorithm to detect negation and its scope. A simple regular expression will not cut it.

4. Taxonomies do not exist for software products and many other businesses

The pre-set taxonomies with rules will not exist for non-standard products or services. This is particularly problematic for the software industry, where each product is unique and the customer feedback talks about very specific issues

5.Not everyone can maintain rules

In any industry, even if you have a working rule-based taxonomy, someone with good linguistic knowledge would need to constantly maintain the rules to make sure all of the feedback is categorized accurately. This person would need to constantly scan for new expressions that people create so easily on the fly, and for any emerging themes that were not considered previously. It’s a never-ending process which is highly expensive.

And yet, despite these disadvantages, this approach is the most widely used commercial application of Text Analytics, with its roots in the 90s, and no clear path for fixing these issues.

So, are Manual Rules good enough?

My answer to this is No . Most people who use Manual Rules are dissatisfied with the time required to set up a solution, with the costs to maintain it, and how actionable are the insights.

Text Analytics Approach 3. Text Categorization

Let’s bring some clarity to the messy subject of Advanced Text Analytics , the way it’s pitched by various vendors and data scientists.

Here, we’ll be looking at Text Categorization , the first of the three approaches that are actually automated and use algorithms.

What is text categorization?

This approach is powered by machine learning. The basic idea is that a machine learning algorithm (there are many) analyzes previously manually categorized examples (the training data) and figures out the rules for categorizing new examples. It’s a supervised approach.

The beauty of text categorization is that you simply need to provide examples, no manual creation of patterns or rules needed, unlike in the two previous approaches.

Another advantage of text categorization is that, theoretically, it should be able to capture the relative importance of a word occurrence in text. Let’s revisit the example from earlier posts. A customer may be explaining the situation that leads to an issue: «My credit card got declined and the cashier was super helpful, waiting patiently while I searched for cash in my bag.» This comment is not about credit cards or cash, it’s about the behaviour of the staff. The theme «credit card» mentioned in the comment is not important, but «helpfulness» and «patience» is.A text categorization approach can capture it with the right training.

It all comes down seeing similar examples in the training data.

Near perfect accuracy … but only with the right training data

There are academic research papers that show that text categorization can achieve near perfect accuracy. Deep Learning algorithms are even more powerful than the old naïve ones (one older algorithm is actually called Naïve Bayes).

And yet, all researchers agree that the algorithm is not as important as the training data .

The quality and the amount of the training data is the deciding factor in how successful this approach is for dealing with feedback. So, how much is enough? Well, it depends on the number of categories and the algorithm used to create a categorization model.

The more categories you have and the more closely related they are, the more training data is needed to help the algorithm to differentiate between them.

Some of the newer Text Analytics startups that rely on text categorization provide tools that make it easy for people to train the algorithms, so that they get better over time. But do you have time to wait for the algorithm to get better, or do you need to act on customer feedback today?

Four issues with text categorization

Apart from needing to train the algorithm, here are four other problems with using text categorization for analyzing people’s feedback:

1. You will not notice emerging themes

You will only learn insights about categories that you trained for and will miss the unknown unknowns.This is the same disadvantage as manual rules and word spotting has: The need to continuously monitor the incoming feedback for emerging themes, and miscategorized items.

2. Lack of transparency

While the algorithm gets better over time, it is impossible to understand why it works the way it works and therefore easily tweak the results. Qualitative researchers have told me that the lack of transparency is the main reason why text categorization did not take off in their world.For example, if there is suddenly poor accuracy on differentiating between two themes «wait time to install fiber» and «wait time on the phone to set up fiber», how much training data does one need to add, until the algorithm stops making these mistakes?

3. Preparing and managing training data is hard

The lack of training data is a real issue. It’s hard to start from scratch and most companies do not have enough or accurate enough data to train the algorithms.In fact, companies always overestimate how much training data they have, which makes implementation fall below expectations. And finally, if you need to refine one specific category, you will need to re-label all of the data from scratch.

4. Re-training for each new dataset

Transferability can be really problematic! Imagine you have a working text categorization solution for one of your departments, e.g. support, and now want to analyse feedback that comes through customer surveys, like NPS or CSAT.Again, you would need to re-train the algorithm.

I just got off the phone with a subject matter expert on survey analysis, who told me this story: A team of data scientists spent many months and created a solution that she ultimately had to dismiss due to lack of accuracy. The company did not have time to wait for the algorithm to get better over time.

Approach 4: Topic Modelling

Topic modelling is also a Machine Learning approach, but an unsupervised one, which means that this approach learns from raw text.Sounds exciting, right?

Occasionally, I hear insights professionals refer to any Machine Learning approach as «topic modelling», but data scientists usually mean a specific algorithm when they say topic modelling.

It’s called LDA, an acronym for the tongue-twisting Latent Dirichlet Allocation. It’s an elegant mathematical model of language that captures topics (lists of similar words) and how they span across various texts.

Example of topic modelling in action

Here is an example of applying topic modelling to beer reviews:

1. The input are reviews of various beers
2. A topic is a collection of similar words like coffee, dark, chocolate, black, espresso
3. Each review is assigned a list of topics.In this example, The Kernel Export stout London has 4 topics assigned to it.

The topics can also be weighted. For example, a customer comment like « your customer support is awful, please get a phone number «, could have weights and topics as following:

• 40% support, service, staff
• 30% bad, poor, awful
• 28% number, phone, email, call

What’s great about topic modelling

The best thing about topic modelling is that it needs no input other than the raw customer feedback.As mentioned, unlike text categorization, it’s unsupervised. In simple words, the learning happens by observing which words appear alongside other words in which reviews, and capturing this information using probability statistics. If you are into maths, you will love the concept, explained thoroughly in the corresponding Wikipedia article, and if those formulas are a bit too much, I recommend Joyce Xu’s explanation.

There are Text Analytics startups that use topic modelling to provide analysis of feedback and other text datasets.Other companies, like StitchFix for example, use topic modelling to drive product recommendations. They extended traditional topic modelling with a Deep Learning technique called word embeddings. It allows to capture semantics in a more accurate way (more on this in our Part 5).

Why is topic modelling an inadequate technique for feedback analysis

When used for feedback analysis, topic modelling has one main disadvantage:

The meaning of the topics is really difficult to interpret

Each topic does capture some aspect of language, but in a non-transparent algorithmic way, which is different from how people understand language.For instance, how would you interpret the second and the fourth topics for the stout beer in the above example:

Whereas the first and the second topic can be somehow «named» as sweetness and fruitiness, the other two topics are just a collection of words.

Any data scientist can put together a solution using public libraries that can quickly spit out a somewhat meaningful output. However, turning this output into charts and graphs that can underpin business decisions is hard.Monitoring how a particular topic changes over time to establish whether the actions taken are working is even harder.

To sum up, because topic modelling produces results that are hard to interpret because it lacks transparency just like text categorization algorithms do, I do not recommend this approach for analysing feedback. However, I stand by the algorithm as one that can capture language properties fairly well, and one that works really well in other tasks that require Natural Language Understanding.

Approach 5. Thematic Analysis (plus our secret sauce on how to make it work even better)

All of the former approaches mentioned have disadvantages. In the best case, you’ll get OK results only after spending many months setting things up. And you may miss out on the unknown unknowns.

The cost of acting late or missing out on crucial insights is huge! It can lead to losing customers and stagnant growth. This is why, according to YCombinator (the startup accelerator that produced more billion dollar companies than any other), «whenever you are not working on your product you should be speaking to your users».

After Thematic participated in their programme, we’ve been asked for advice three times via a survey, once via a personal email, and also in person. YCombinator also use Thematic to make sense of all the feedback they collect.

When it comes to customer feedback, three things matter:

1. Accurate, specific and actionable analysis
2. Ability to see emerging themes fast, without the need of setting things up
3. Transparency in how results are created, to bring in domain expertise and common sense knowledge

In my research, I’ve learned that the only approach that can achieve all three requirements is Thematic Analysis, combined with an interface for easily editing the results.

Thematic Analysis: How it works

Thematic Analysis approaches extract themes from text, rather than categorize text. In other words, it’s a bottom-up analysis. Given a piece of feedback such as «The flight attendant was helpful when I asked to set up a baby cot», they would extract themes such as «flight attendant», «flight attendant was helpful», «helpful», «asked to set up a baby cot «, and» baby cot «.

These are all meaningful phrases that can potentially be insightful when analyzing the entire dataset.

However, the most crucial step in a Thematic Analysis approach is merging phrases that are similar into themes and organizing them in a way that’s easy for people to review and edit. We achieve this by using our custom word embeddings implementation, but there are different ways to achieve this.

For example, here is how three people talk about the same thing, and how we at Thematic group the results into themes and sub-themes:

Advantages and disadvantages of Thematic Analysis

The advantage of Thematic Analysis is that this approach is unsupervised, meaning that you do not need to set up these categories in advance, do not need to train the algorithm, and therefore can easily capture the unknown unknowns.

The disadvantages of this approach are that it’s difficult to implement correctly. A perfect approach must be able to merge and organize themes in a meaningful way, producing a set of themes that are not too generic and not too large. Ideally, the themes must capture at least 80% of verbatims (people’s comments). And the themes extraction must handle complex negation clauses, e.g. «I did not think this was a good coffee».

Who does Thematic Analysis?

Some of the established bigger players have implemented Thematic Analysis to enhance their Manual Rules approaches but tend to produce a laundry list of terms that are hard to review.

Traditional Text Analytics APIs designed by NLP experts also use this approach. However, they are rarely designed with customer feedback in mind and try to solve this problem in a generic way. For example, when we tested Google and Microsoft’s APIs we found that they are not grouping themes out of the box.

As a result, only 20 to 40% of feedback is linked to top 10 themes: only when there are strong similarities in how people talk about specific things. The vast majority of feedback is uncategorized meaning that you can not slice the data for deeper insights.

At Thematic, we have developed a Thematic Analysis approach that can easily analyze feedback from customers of pizza delivery services, music app creators, real estate brokers and many more. We achieved this by focusing on a specific type of text: customer feedback, unlike NLP APIs that are designed to work on any type of text. We have implemented complex negation algorithms that separate positive from negative themes, to provide better insight.

Our secret sauce: Human in the loop

Each dataset, and sometimes even each survey question, gets its own set of themes, and by using our Themes Editor, insights professionals can refine the themes to suit their business.For example, Thematic might find themes such as «fast delivery», «quick and easy», «an hour wait», «slow service», «delays in delivery» and group them under «speed of service». One insight professional might re-group these into «slow» and «fast» under «speed of service», another into «fast service»> «quick and easy», and «slow service» -> «an hour wait», » delays in delivery «. It’s a subjective task.

I believe more and more companies will discover Thematic Analysis, because unlike all other approaches, it’s a transparent and deep analysis that does not require training data or time for crafting manual rules.

What are your thoughts?

Which approach is right for you?

We’ve created a cheat sheet which lists the text analytics approaches, check it out here below or view the larger version here.

Want to try a demo of Thematic? I’d love to have a chat with you, book a time with me here.

.