Chapter 250 Incredibly clever idea

Specifically, Lin Hui changed the paper written by Eve Carly at that time.

Objectively speaking, in fact, Lin Hui's changes in that paper at that time were not very large in terms of generative text summarization.

Lin Hui just added some content.

But what Lin Hui added is almost all the essence.

Through Lin Hui's supplementary content, Eve Carly learned more about how Lin Hui mastered the text summarization technology in Nanfeng APP.

Lin Hui took many ingenious methods to build a generative text summarization algorithm.

Whether it is designing appropriate model architecture and training strategies based on deep learning technology.

Still using the idea of ​​transfer learning, a generative automatic text summary algorithm based on the pre-training model was proposed.

Or complete content representation and weight calculation through unsupervised methods.

These are things that Eve Carly has never thought of before, or has never understood deeply.

A Ph.D. in a related field actually has something that he didn't realize before?

It sounds weird, but it's true.

As the saying goes, there is a sequence of learning, and there is a specialization of skills.

There is nothing unacceptable about falling behind others for a while.

And Eve Carly is sure that her situation is definitely not an isolated case.

Eve Kali felt that what Lin Hui added might not have been something she had not thought of.

Many other researchers may not have thought of it either.

Lin Hui proposed some new insights not only compared with traditional text summarization research.

Even what Lin Hui tinkered with can be called a brand new idea for the entire NLP direction.

Anyway, Eve Carly thinks these ideas are very wonderful and can even give people a kind of enlightenment effect.

The reason for this effect is largely due to the fact that most text summarization researchers previously studied extractive text summarization.

Although both extractive text summarization and generative text summarization are text summarization.

But the transition from the former to the latter involves a process of transformation in thinking.

In many cases, most researchers in traditional text summarization, that is, researchers who study extractive text summarization, are often influenced by preconceptions and do not fully understand generative text summarization.

For example, take the pre-training proposed by Lin Hui when he was working on generative text summarization.

Ordinarily, this thing is not a profound concept.

The so-called pre-training is not difficult to understand, it is nothing more than rough processing of the data for training the model.

But this thing is harder to think of.

In the past, Eve Kali did not use pre-training when tuning extractive text summarization.

In most cases, training is performed directly.

The pre-training step is not applied.

Follow Lin Hui’s additions in the paper.

The common practice of pre-training is to put together a large amount of low-cost collected training data.

Then use a certain type or type of specific pre-training method to learn the common features of these training data.

The commonalities are then transplanted into task-specific models.

Then use a small amount of annotated data in relevant specific fields for more detailed adjustments.

After completing this process, future models for practical applications only need to start from commonalities.

Then just learn the special parts of a specific task.

It is roughly similar to the process of finding general solutions to some equations first and then finding specific solutions.

It sounds quite abstract.

It's actually not that profound.

When it comes to machine learning, no matter how advanced it is.

In essence, they are basically imitating people.

In this case, we often just need to understand how people deal with problems.

You can understand the ideas or methods of machine learning to deal with problems.

Usually when we are learning something.

Perhaps our original intention is to learn everything we want to learn at once.

But due to limited study time, numerous academic tasks or various other objective factors.

When actually studying, it is difficult to learn all the knowledge in one go.

In this case, how do some people who are good at learning learn?

What these people may adopt when learning is to first understand the common content of the knowledge they want to learn.

Then spend time on some "difficult diseases".

Although this approach seems a bit "lazy".

But more than half of mankind’s wisdom comes about because of laziness.

It is undeniable that this seemingly lazy way of learning is full of wisdom.

At least from an efficiency perspective, this approach is commendable.

After all, except for extremely special subjects such as medicine.

80% of the knowledge involved in most fields can find commonalities.

After finding commonalities, then solve the other 20% of complex knowledge.

This is undoubtedly a more labor-saving way of thinking.

Introducing pre-training in natural language processing, a typical direction of machine learning.

It is undoubtedly equivalent to "transplanting" a special technique that some outstanding students use in their studies.

This idea is undoubtedly very clever.

The idea is certainly very clever.

But just like Li Ku on the roadside.

Why has no one tried this clever idea before?

Eve Carly feels that no one may have thought about this aspect.

But others failed without exception.

When it comes to knowledge acquisition, perhaps most people also know that it can save effort to get 80% of the common knowledge first and then the other 20%.

But judging from her past studies, Eve Carly feels that there are very few people around her who can first find out the commonalities of 80% of the knowledge and then overcome the difficulties.

There is even no one except the top academics in Eve Carly's eyes who can do this.

How many top academics can there be in Eve Carly's eyes? It can be said to be very few.

In other words, this very wise approach of first getting 80% of the common knowledge and then getting the other 20% is actually rarely used.

It's obviously the easier way.

Why don't many people do this?

Eve Carly thinks the main reasons are:

——Most people are not good at finding commonalities in knowledge.

If they are not good at finding commonalities in knowledge, some people will try to find commonalities in knowledge.

But in actual operation, it is completely unrealistic to find the commonality of 80% of knowledge.

You may only be able to find 30%, 20% or even less commonalities in knowledge.

As a result, these people not only failed to find the commonality of subject knowledge.

On the contrary, when looking for commonalities, some other originally ordinary content was unknowingly alienated and turned into "non-common knowledge" in the eyes of these people.

However, non-common knowledge has become more troublesome knowledge in the minds of these people who are trying to find commonalities.

These are not particularly difficult knowledge in the first place, but with the debuff of psychological suggestion.

The efficiency is even lower than when no commonality is found.

In this way, people who have not found commonalities may become the content that those who try to find commonalities need to spend a lot of time to overcome.
Chapter completed!