Lessons learned
In the PPSA 2025 report, the experts in TRADEIL and MP Data Technologies describe the incorporation of artificial intelligence into the processing of data from in-line controls. The increase in ultrasound data required a faster search for defects. On a sample from 129 The authors report that they are likely to detect real cracks 95%; separate lines without field feedback are considered. These results are for the inspection described and not for all possible pipelines.
The authors also note that a large number of false concerns can reduce the time-use. The following is how this system can be evaluated from an engineering point of view.
Division of tasks makes the result verifiable
Automatic processing involves several different operations. The program first looks for a suspicious area, then classify it and evaluates the size. Successful implementation of the first operation does not prove the accuracy of subsequent operations. For example, the system can safely select welds but mistakenly refer its local response to a crack.
Therefore, the test should be separately answerable to questions: how many defects were missed, how many of the correct sites were analysed and how correctly defined the characteristics of the detected objects. For the developer, this means that intermediate results must be maintained. The resulting colour mark without a reference signal makes it difficult to remove errors and improve the model.
Why the overall proportion of correct answers doesn't say much
Most data in the long pipeline do not contain a specific defect. An algorithm that almost always chooses a "no crack" class can show a high percentage of correct answers and be useless. For rare events, sensitivity to the defects of the required size and pressure from false operation is more important.
The report should indicate the sample composition, the number of independent objects and how to obtain reference marks. Single issue — Undefined cases where even experts disagree. They cannot be overlooked in the evaluation: it is these signals that often present the main difficulty in operating the system.
Training and testing should be divided by origin
The surrounding parts of the same recording are usually similar. If some of the fragments are in training and the neighbouring — In fact, the algorithm meets familiar conditions. The assessment may be better than the result on a new pipeline, other instrument or other registration setting.
More meaningful verification divides data for independent sites or surveys. Tests on unusual forms of signal, pollution, channel passes and speed changes are further useful. Such an organization does not guarantee universality, but helps to understand what differences the model really carries and where it needs separate expertise.
False alarm has a measurable cost
Each candidate requires specialist time: to open data, compare channels, check the link and record the decision. Even a fast automatic view can increase the overall length of work if the number of candidates is too large. So the computation rate on the computer is not equal to the output rate of the verified report.
It is easy to estimate the capacity of the entire route: data production, machine analysis, expert verification, re-examination of disputed objects and output. Both effects should be shown when changing the threshold of sensitivity — Change in number of omissions and change in the expert load. The choice of threshold then becomes a transparent engineering compromise.
A person must see the basis for a decision
The expert needs more than just a foretold class and a imputed percentage of confidence. It is more useful to show both the reference signal, the adjacent channels, the position of the seam and the signs on which the conclusion is based. The number of model confidences should not be automatically understood as the probability of a crack: this requires a separate calibration check.
It is particularly important to provide a data route outside the known field of application. The system may mark them as requiring manual consideration rather than being forced into the familiar class. The rejection of automatic detention in a mixed situation was a quality process, not a sign of its weakness.
How to make a useful feedback
The expert's correction of automatic tag becomes a valuable learning example only if the context is maintained. It should be recorded what has been changed: the presence of an object, its class, boundaries or dimensions. It is desirable to distinguish between a certain correction and a controversial case that requires further information.
Otherwise, the next version of the model will start learning from mixed solutions without understanding their reliability. Selective re-marking by an independent specialist is useful. It not only shows the mistakes of the program, but also shows the consistency of the human process. If experts are to be systematically separated, the classification rules need to be clarified first. The expansion of the training base without such a step can scale up the ambiguity rather than eliminate it.
Once implemented, verification is ongoing
The algorithm version, pre-treatment parameters and the composition of learning data affect the result. The old records can be updated to obtain new estimates. Changes should therefore be accompanied by a test sample, a version log and an explanation of the significant differences. Without this, comparisons of surveys across years become untransparable.
In the editorial sense of OKB, the usefulness of artificial intelligence is determined by the quality of the entire chain of evidence. Automation helps to work with a large amount of measurement, but does not replace metrology, understandable problem statement and responsibility for final interpretation. These are the elements that turn the research algorithm into an industrial tool.
Engineering output
The AI should be evaluated by independent verification, pass, false alarms and complete work-intensive analysis. The model result shall remain traceable until the reference measurements are made.
Sources and further reading
- PPSA Technical Report 2025open PDFTRAPIL / MP Data Technologies — How AI Can Reshape the Data Analysis Process of ILI Tools ↗
The article was originally written in Russian. It summarises the cited sources. For the original text and illustrations, consult the publications listed below.