Generating Abstractive summaries

[nader01]

I have tried a few models for Abstractive summaries, but neither are producing a reasonable form of a summary, which I define as a single, short paragraph. You can see the various models used so far in the code. The one in use in the code below is facebook/bart-large, which is producing a summary, longer than the original article.

Original Article: 485 words
Summary: 685 words

I would appreciate your help with model selection or alternate approach to summarization.

import torch
from transformers import pipeline

file = open("Data/article.txt", "r")
article = file.read()

#  model_name = 'google/pegasus-newsroom'
#  model_name = 'Artifact-AI/led_base_16384_billsum_summarization'

model_name = 'facebook/bart-large'

summarizer = pipeline('summarization', model=model_name,
                      max_new_tokens=1024,
                      truncation=True, framework='pt')
summary = summarizer(article)
print(summary)

original Article:
With extremely cold temperatures gripping much of the United States, travelers may be wondering how their planes are made safe for flying. The short answer: Along with regular deicing of planes before take-off, airlines put their aircraft through rigorous testing in some of the coldest places on the planet before they are ever pressed into service. CNN has previously followed a team of Airbus engineers, mechanics and test pilots. Their mission was to see how their test plane, a long-range A350 XWB, withstood extreme cold. The Airbus team left the warmth and comfort of their Toulouse, France, base to perform various extreme-weather trials on the test plane in frigid Iqaluit, the capital of Canada’s eastern Arctic territory of Nunavut. The ground and in-air tests included operating the A350 XWB in temperatures reaching down to -18 Fahrenheit (-28 degrees Celsius), thrust-reversed tests with snow and a local flight test. From cold to hot All new plane models must be tested in extreme environmental conditions, from freezing to intense heat. “Coming to an extreme place means we can break everything,” Airbus’ head of flight operations Pedro Dias told local reporters. Extreme cold affects various parts of an aircraft in different ways. Metals, such as steel and aluminum, contract at different rates. Lubricants may lose their viscosity, creating friction and wear issues for moving parts. Meanwhile plastic and rubber parts could become brittle. The cold-weather tests in Canada came only days after the MSN3 test plane completed high-altitude test in Bolivia. Next stop was Qatar for hot-weather testing. ‘Premier cold-weather test site’ For years Iqaluit, located on Baffin Island, has marketed itself as “a premier cold-weather test site.” Airbus has tested there since the 1990s, while the airport has also hosted other civilian and military aircraft makers such as Boeing, Dassault and Eurocopter to perform similar trials. But cold weather test sites also appear in unlikely places. In April 2010, Boeing chose McKinley Climatic Laboratory in Florida as the location for extreme-weather testing on its 787 Dreamliner. In a test chamber the aircraft was given a “cold soak” and exposed to temperatures as low as -45 degrees Fahrenheit (-42.7 degrees Celsius) for hours. Later, it had to endure temperatures as high as 115 degrees Fahrenheit (46 degrees Celsius), also for hours. “These tests help us establish that our customers will get airplanes that work for them in all of the climates in which they operate around the globe (and in all seasons),” Randy Tinseth, vice president of marketing at Boeing Commercial Airplanes, said in a news release. But extreme-weather trials are also affected by unpredictability of Mother Nature. Initially scheduled for five days, Airbus’ tests in Iqaluit were cut short, not because of an impending winter blizzard but because of a balmy temperature of -18 degrees Celsius – too warm for the cold weather testing. This article was first published in February 2014 and was updated in January 2023.

[{‘summary_text’: 'With extremely cold temperatures gripping much of the United States, travelers may be wondering how their planes are made safe for flying. The short answer: Along with regular deicing of planes before take-off, airlines put their aircraft through rigorous testing in some of the coldest places on the planet before they are ever pressed into service. CNN has previously followed a team of Airbus engineers, mechanics and test pilots. Their mission was to see how their test plane, a long-range A350 XWB, withstood extreme cold. The Airbus team left the warmth and comfort of their Toulouse, France, base to perform various extreme-weather trials on the test plane in frigid Iqaluit, the capital of Canada’s eastern Arctic territory of Nunavut. ‘Premier cold-weather test site’ For the past two years, the Airbus MSN3 test plane has been flown from Toulon, France to the Arctic, where it has been subjected to a variety of extreme conditions. In a test chamber the aircraft was given a “cold soak” and exposed to temperatures as low as -45 degrees Fahrenheit (-42.7 degrees Celsius) for hours. Later, it had to endure temperatures as high as 115 degrees Fahrenheit (46 degrees Celsius), also for 24 hours. The ground and in-air tests included operating the A350XWB in temperatures reaching down to -18 Fahrenheit (-28 degrees Celsius, thrust-reversed tests with snow and a local flight test. Extreme cold affects various parts of an aircraft in different ways. Metals, such as steel and aluminum, contract at different rates. Lubricants may lose their viscosity, creating friction and wear issues for moving parts. Meanwhile plastic and rubber parts could become brittle. “Coming to an extreme place means we can break everything,” Airbus’ head of flight operations Pedro Dias told local reporters. From cold to hot All new plane models must be tested in extreme environmental conditions, from freezing to intense heat. But cold weather test sites also appear in unlikely places. In April 2010, Boeing chose McKinley Climatic Laboratory in Fort Lauderdale, Florida, as the location for extreme- weather testing on its 787 Dreamliner. The Boeing 787-9 Dreamliner was tested there for the first time in January 2010, when it was subjected to extreme heat and cold conditions. Next stop was Qatar for hot-weather testing. These tests are important to Boeing because they help ensure that the 787 will be able to fly in all weather conditions. The test plane was also used in a test flight in the Mojave Desert in California last year. "These tests help us establish that our customers will get airplanes that work for them in all of the climates in which they operate around the globe (and in all seasons),” Randy Tinseth, vice president of marketing at Boeing Commercial Airplanes, said in a news release. ‘Unpredictable’ weather conditions But the cold weather trials are also affected by unpredictability of Mother Nature. Initially scheduled for five days, Airbus‘ tests in Iqaruit were cut short, not because of an impending winter blizzard but because of a balmy temperature of -18 degrees Celsius – too warm for the cold cold weather testing. The cold weather tests in Canada came only days after the MSN 3 test plane completed high-altitude test in Bolivia. « This article was first published in February 2014 and was updated in January 2023. The original version was published in January 2014. 'Premier’ cold-winter test site. Airbus has tested there since the 1990s, while the airport has also hosted other civilian and military aircraft makers such as Boeing, Dassault and Eurocopter to perform similar trials. However, Airbus has been the only aircraft maker to test in the Arctic since the 1970s. '}]

[wsfung2008]

Try using a smaller max_new_tokens.

[nader01]

I tried various models and found some interesting results.
Model ‘sshleifer/distilbart-cnn-12-6’ is the default summarization model when using PyTorch. See:

Pipelines.

The default model, using pyTorch, can be used as follows:

from transformers import pipeline

file = open("Data/article.txt", "r")
article = file.read()

summarizer = pipeline('summarization')
summary = summarizer(article, max_length=130, min_length=30, do_sample=False)
print(summary)

I also could use ‘facebook/bart-large-cnn’, which is another good Abstractive summarization model with similar performance. Using this model, you need to pass-in the model name to the pipeline using ‘model=’ parameter.

Both models result in good summaries with reasonable length and in about 30s on my PC which has no GPU.

Results follow:

Model: facebook/bart-large-cnn, max_length=130, min_length=30

[{‘summary_text’: " All new plane models must be tested in extreme environmental conditions, from freezing to intense heat . Airbus’ long-range A350 XWB tested in frigid Iqaluit, the capital of Canada’s eastern Arctic territory of Nunavut . Boeing chose McKinley Climatic Laboratory in Florida as the location for extreme-weather testing on its 787 Dreamliner ."}]

Model: facebook/bart-large-cnn, max_length=200, min_length=100

[{‘summary_text’: ‘All new plane models must be tested in extreme environmental conditions, from freezing to intense heat. Extreme cold affects various parts of an aircraft in different ways. Boeing chose McKinley Climatic Laboratory in Florida as the location for extreme-weather testing on its 787 Dreamliner. In a test chamber the aircraft was exposed to temperatures as low as -45 degrees Fahrenheit (-42.7 degrees Celsius) for hours. Later, it had to endure temperatures as high as 115 degrees Fahrenheit (46 degrees Celsius), also for Hours.’}]

Model: sshleifer/distilbart-cnn-12-6 Max_lenth=130, min_length-30

[{‘summary_text’: ‘All new plane models must be tested in extreme environmental conditions. Extreme cold affects various parts of an aircraft in different ways. Boeing chose McKinley Climatic Laboratory in Florida as the location for extreme-weather testing on 787 Dreamliner.’}]

Model: sshleifer/distilbart-cnn-12-6 Max_lenth=200, min_length-100

[{‘summary_text’: " All new plane models must be tested in extreme environmental conditions, from freezing to intense heat . Airbus’ long-range A350 XWB test plane was tested in frigid Iqaluit, the capital of Canada’s eastern Arctic territory of Nunavut . Boeing chose McKinley Climatic Laboratory in Florida as the location for extreme-weather testing on its 787 Dreamliner in 2010 . The cold-weather trials are also affected by unpredictability of Mother Nature, such as an impending winter blizzard ."}]