Attitude Extraction Advances

OHIF Import Measurements

2026-03-20

The most relevant question raised in OHIF issues: https://github.com/OHIF/Viewers/issues/1800 Version 3.10.2

Figure The overall limitation of the single route per mode.

Forming Working Prototype. We first extract trackedMeasurements and use it as a reference format of data we going to deal with when importing measurments:

const measurements = measurementService.getMeasurements();

Click to expand reference format

```json { "uid": "11570db5-d2a8-4868-a6e7-6017bc230710", "SOPInstanceUID": "1.2.826.0.1.3680043.8.274.1.1.8323329.421317.1748947541.188095", "FrameOfReferenceUID": "1.2.826.0.1.3680043.8.274.1.1.8323329.421317.1748947541.188070", "points": [ [ 55.84648437499999, 305.7054700285197, 4 ], [ 97.74101562499999, 7.416407528519642, 4 ], [ 176.22343749999993, 170.52578252851958, 4 ], [ -22.635937500000068, 142.59609502851967, 4 ] ], "textBox": { "hasMoved": false, "worldPosition": [ 97.74101562499999, 156.56093877851964, 4 ], "worldBoundingBox": { "topLeft": [ 139.63554687499993, 198.4554700285196, 4 ], "topRight": [ 308.889453125, 198.4554700285196, 4 ], "bottomLeft": [ 139.63554687499993, 298.66718750000007, 4 ], "bottomRight": [ 308.889453125, 298.66718750000007, 4 ] } }, "isLocked": false, "isVisible": true, "metadata": { "toolName": "Bidirectional", "viewPlaneNormal": [ 0, 0, -1 ], "viewUp": [ 0, -1, 0 ], "FrameOfReferenceUID": "1.2.826.0.1.3680043.8.274.1.1.8323329.421317.1748947541.188070", "referencedImageId": "wadors:http://127.0.0.1/OrthancNginx/dicom-web/studies/1.2.86.76547135.7.115532.20160916104421/series/1.2.826.0.1.3680043.8.274.1.1.8323329.421317.1748947541.188084/instances/1.2.826.0.1.3680043.8.274.1.1.8323329.421317.1748947541.188095/frames/1", "cameraFocalPoint": [ 176, 176, 4 ], "sliceIndex": 4 }, "referenceSeriesUID": "1.2.826.0.1.3680043.8.274.1.1.8323329.421317.1748947541.188084", "referenceStudyUID": "1.2.86.76547135.7.115532.20160916104421", "referencedImageId": "wadors:http://127.0.0.1/OrthancNginx/dicom-web/studies/1.2.86.76547135.7.115532.20160916104421/series/1.2.826.0.1.3680043.8.274.1.1.8323329.421317.1748947541.188084/instances/1.2.826.0.1.3680043.8.274.1.1.8323329.421317.1748947541.188095/frames/1", "frameNumber": 1, "toolName": "Bidirectional", "displaySetInstanceUID": "1ab64ffd-230c-d0aa-6161-00fa510e31b8", "label": "", "displayText": { "primary": [ "L: 301 mm", "W: 201 mm" ], "secondary": [ "S: 0 I: 4" ] }, "data": { "imageId:wadors:http://127.0.0.1/OrthancNginx/dicom-web/studies/1.2.86.76547135.7.115532.20160916104421/series/1.2.826.0.1.3680043.8.274.1.1.8323329.421317.1748947541.188084/instances/1.2.826.0.1.3680043.8.274.1.1.8323329.421317.1748947541.188095/frames/1": { "length": 301.21672688578593, "width": 200.81115125719057, "unit": "mm", "widthUnit": "mm" } }, "type": "value_type::shortAxisLongAxis", "source": { "uid": "53d521c7-de53-3c20-1134-83cd2f1b77d7", "name": "Cornerstone3DTools", "version": "0.1" }, "modifiedTimestamp": 1773825233, "isDirty": true, "isSelected": true } ```

Then we use data to form the following request to measurement service (addRawMeasurement).

Important: We need to place this code after cornerstone is avaialble to display related content.

Next crucial problem is to reshape measurements and distribute them across addRawMesurements input parameters.

First, we need source that is aligned with the one in measurements by guid.
The same is for further mappings, i.e. sourceMappings.

For that, measurementService provides related API:

 const CORNERSTONE_3D_TOOLS_SOURCE_NAME = 'Cornerstone3DTools';
 const CORNERSTONE_3D_TOOLS_SOURCE_VERSION = '0.1';

 const source = measurementService.getSource(
     CORNERSTONE_3D_TOOLS_SOURCE_NAME,
     CORNERSTONE_3D_TOOLS_SOURCE_VERSION
 );

 const mappings = measurementService.getSourceMappings(
     CORNERSTONE_3D_TOOLS_SOURCE_NAME,
     CORNERSTONE_3D_TOOLS_SOURCE_VERSION
 );

Because data don’t have cachedStats. Do we have to manually register. We can assign default values {}. And it works!

However it should not be empty, as in UI you won’t see info about H/Width of the measurement.

  const annotation = {
    data: {
        // Manually add.
        cachedStats: {}
        ...
    }
  }

Therefore we need to put data with key equals to referencedImageId at cachedStats as follows:

  // That duplicates the metadata content but would need for cornerstone to display.
  // See later: requires added imageId.
  'wadors:http://127.0.0.1/OrthancNginx/dicom-web/studies/1.2.86.76547135.7.1323518.20210710080000/series/1.2.826.0.1.3680043.8.274.1.1.8323329.418900.1748947477.387488/instances/1.2.826.0.1.3680043.8.274.1.1.8323329.418900.1748947477.387555/frames/1': {
length: 301.21672688578593,
width: 200.81115125719057,
unit: 'mm',
widthUnit: 'mm',
  },

Register annotation UID as well * This could be done as: js const annotation = { // annotationUID: toolData.annotation.annotationUID, annotationUID: '11570db5-d2a8-4868-a6e7-6017bc230710', }

Next, is Problem on cornerstone side with rendering Bidirectional measurement.

By exploring code, this step involves instantiation of the measurement (copy creation). I belive this process not entirely smooth and causes futher misalignments at rendering stage resulted in exceptions.
- Then you get an exception:
```
Error: getTargetIdImage: targetId must start with "imageId:" or "volumeId:"
```
  because key in cached data should start with “imageId:” …
- For some reason, cornerstone attempt to draw it with circular tool. Eitherway the problem for now is that no points (for some reason) is available to render this content.
  
  When this call happens.
```
const svgDrawingHelper = getSvgDrawingHelper(element);
```
  at drawHandles.js
```
drawHandle(svgDrawingHelper, annotationUID, handleGroupUID, handle, options, i);
```
  handle is None.
  
  After further investigations I found that activeHandleCanvasCoords is undefined, which is not equals to None (Tools/dist/esm/tools/annotation/BidirectionalTool.js)
  
  Expects activeHandleIndex that at least should be null (not undefined)
  
  The solution: cornerstone expects activeHandleIndex: null, so we have to add so in handles.
```
const annotations = {
  handles: {
      // Important for handling with cornerstone for Bidirectional type.
      activeHandleIndex: null
  }
}
```

And that’s it 🥳

The function:

function importMeasurement(measurementService, measurement) {
  const CORNERSTONE_3D_TOOLS_SOURCE_NAME = 'Cornerstone3DTools';
  const CORNERSTONE_3D_TOOLS_SOURCE_VERSION = '0.1';

  const source = measurementService.getSource(
    CORNERSTONE_3D_TOOLS_SOURCE_NAME,
    CORNERSTONE_3D_TOOLS_SOURCE_VERSION
  );

  const mappings = measurementService.getSourceMappings(
    CORNERSTONE_3D_TOOLS_SOURCE_NAME,
    CORNERSTONE_3D_TOOLS_SOURCE_VERSION
  );

  const annotationType = measurement.metadata.toolName;

  
  // TODO. Add label restoration.
  const annotation = {
    annotationUID: measurement.uid,
    metadata: measurement.metadata,
    data: {
      cachedStats: measurement.data,
      handles: {
        activeHandleIndex: null,
        points: measurement.points,
      },
    },
  };

  const matchingMapping = mappings.find(m => m.annotationType === annotationType);

  measurementService.addRawMeasurement(
    source,
    annotationType,
    { annotation },
    matchingMapping.toMeasurementSchema
  );
}

I missed the train but got no delays: story of traveling from north to south in the UK

2026-03-20

🖼️ Unexpected traveling plan from Bolton to London: 1-2 Initial Route ⚫, but I took 3-4 🔴

Manchester Picadilly station represents one of the larges hub that connects South-West UK county. Once you calling Manchester, no matter coach or train, you instantly becoming aware of it. I used to callig that place for a long time, and it used to be journeys from far away: Newcastle Upon Tyne, and so far London (Euston train station) in this list. However, there is a known psychological effect that finds manipulative applications in certain industries and involves repretitive series of actions with the same outcome.
Something that you’re doing quite often through the same path makes you a biased person in the particular domain area. This is what happened and you may find yourself the same way …

I remember once traveling back to the south of the UK with no delays and rush. The autopilot mode on traveling, accompanied with the relaed app assistances is what basically relaxes you. Sometemies you may find it too suspicious and ended up being confused by thinking too much or by raising a wise questions. The one question that came up to me so far, on my two-leg trip from Bolton (Manchester suburban train) to London (Euston train station). And the question was as follows:

How the hell the app notifies me to take a train that goes absolutely different direction?!

I was too lucky to end up into this right at the arrival time of the expected train 🚄. Being confused I asked staff on how to call I should call Manchester with this train. That’s exacly how my journey of this post begins, accompanied by the following response:

👧 Well, to call that place you have to change the platform an take the next train.

Believing in confidence of the professionals helps you to cope. So I did. Only and after a while I realise when the bias works against me: the station is intented to be called was different… Different city, which was Blackburn

That opens the story of how I missed the train but just not entering in it. Accepting my failure, It was decided to came to the same officer for the related recommendations on picking another traveling route. As it usually happens in the UK, the officer treat the overall action happened with her assistance as a problem that bothrers her too. She was intended to help me and later on suggester another route: call Preston and travel from there to London.

So I did:

📸 Preston train station

The good aspect is that traveling was free of charge for me 🥳 But then you may wonder: how you can get to the South (London) with the train that has prescribed seats in tickets? Well, in the UK they have so-called reserved train coaches. They may refer to it as “Universal” and therefore several trains may have “U’ coaches as the one below:

📸 I took this “U” coach of the train, where anyone is free to take a sit wheereever they wish!

In the end, it took me nearly extra 20 minutes delay with this trip to London. And the conclusion out of it is as follows:

Nonetheless the whole story and the way staff help makes me happy about service in the UK that is relatd to trains.

No matter how effective your trip in your mind in terms of train logistics, you may still ended up being travel in opposite directions!

Visa Granted but Working permission Refused: 76 days of limbo or ATAS adventures

2026-03-20

Picture: Observation wheel of the new spot 🎡 or such an adventure traits?

When contranct is about to over or any other circumstaces sings to its termination, it becomes a time of personal iterest in eligibility of your further rights to work and stay in country. In some cases it might be just a simple switch to another position, but in others causes to issue a new VISA.

For the UK, Skilled worker 💼 is a one of a such a common ways of being employed for foreigners. Despite the certain benefits, the major drawback of this option is a limitation of your rights. All your rights and eligibility to stay in country is restricted to the particular employer. Depending on the industry or academia as a type of your employment, you may a have a different routes of VISA issuing.

Academia tends to a be a way more tough route due to the additional approvements, necessary for issuing VISA. And one of them is ATAS, or Academical Technology Approving Scheme. In some particular cases, both PhD students and post-doc resarchers might need to apply for this certificate. Being relatively simple in submission, the overall ATAS application reviewing process may be counted as «mini-version of the US 🇺🇸 / Canada 🇨🇦 VISA reviewing process». It literally prescribes you (see “How long application take” section) await for 6 weeks in general with no fast-tracking options. So that, now you aware of applying as early as it possible, with 3 … no, how about 6 months in advance recommendation! 🤯 Even more, from the form above you’re also aware of canceling any settiling plans while awaiting for the result decision.

Unfoturnately and as long as the form says, being NLP researcher in AI and with nationality mentioned in list, I found that I need this ceritficate. So far, I got 2 and a half months remaining of my eligibility to stay, and that becomes a story of almost 🔥76 days of limbo🔥 awaitance for submitting a new Skilled worker VISA application. And that time has become a series of lessons learned a hard way.

Back more than 2 months ago I submitted ATAS by the beginning of October 2023 with further issues corections within following days of the same week. So that quick response from ATAS department may cause you a relief of their quick interest and your application consideration. Even more, that was my second time application, which makes me at least more-or-less proven applicant. That means I knew for how long to await for the decision so I can barely arrange the plans. To clarify, you’re not eligible for submitting more that 1 ATAS application for academia positions in the UK at the time.

After a 6 weeks of pacient awaitance, the patience has become no longer a working strategy. And that point where you may start investigate how others cope it. The latter become a reason I was running into the following case, and that makes you guess you’re not the only there with long-term awaitance. Your overall goal is to obtain Certificate-of-Sponsorship, or CoS in short. The latter prescribes your starting day of the contract, and ending day (if applicable and in the case of the fixed term contract). You may strongly recommend to start being active after 6 weeks and even after 2 months of awaitance! Your HR is the only rescue team from this, and you should get the most out of conversation with them.

Sadly to say, I found the related team in a very improfessional way. Through the journey of assisting and finding a way for issuing documents without ATAS took me the whole month! Throgh this, you realise on how useless the calls to HR and how you should form messages to reach the team.

So that end of the story that was sorted just in 7 days of my remaining eligiblity to stay in the UK:

HR team found a way on how to issue Certificate of sponsorship without ATAS certificate. 🎊 (credits to the following video)
I’ve submitted my VISA application as switch-to Skilled Worker VISA under the fast-tracking service within the UK
Got the confirmation from the Home Office, followed by issuing and delivering a new Biometric Residene Permit (BRP) within just 5 working days 🏎️

Credits to the VISA fast tracked option 🎊

Breathe out, right❓ Yes, and how was that afterwards:

I got a letter with the REJECTED decision from the ATAS department.
Reaching out UK Visas and Immigration service (UKVI) 🤙 I’ve been informed to search for the local legal to contact the Legal Center for Immigration Enquiries after a call to UKVI (basically they share only information from GOV.UK)

I called 🤙 and booked an appointment at the local Legal Center which is at 14 West Rd, Newcastle upon Tyne NE4 9HB, the next day and been informed to bring all the necessary documents for case explanation.

During my apporintment, the advisor at legal center has navigate me straightaway to the Sponsor guidance part-2 at GOV.UK, and provide the following and general comments to this case:

Legal advisor: If the home office accepts and grants with rights to be in UK under certain conditions, covered by “employment allowed” section then the overall process of issuing an ATAS certificate is no longer applicable. The latter means that it may have status “Refused” which could be treated as “Not Applicable”.

In other words, if your employer accepts the conditions that are later on confirmed by the Home Office, then it is no longer your responsibility as well as eligibility to have a wrong right to work in the UK.

The decision of the Home Office is final, so anything that prevents it is likely no longer applicable. For any further circumstances on that, the employer’s organization is taking responsibility.

What's new in AREkit-ss Sources Sampler 0.24.0 Release

2026-03-20

Sampling of the contexts with mentioned sentiment relations in it has become even simple to use rather it was before!

So far we work hard on the following features that make the application of this process as well as the application of the obtained results even more handy. Here is the upcoming features and changes in AREkit-ss 0.24.0:

✔️: Support of the sqlite3 for storing the results

✔️: Refactored backend of the translator, enhanced translation with masked spans in texts

✔️: Dynamic import and sources registration, which is basically means errors in one source won’t affect the overall behavior of the application.

✔️: Support for the following collections: NEREL and BioNEREL

✔️: Other bug fixes aimed on better adaption of the result texts for LLM, optional entities masking flexibilities

🌟 AREkit “double-s”: https://github.com/nicolay-r/arekit-ss

🌟 AREkit core: https://github.com/nicolay-r/AREkit

The Recent Advances in LLM Alignment With Manually Annotated Relations in Mass-Media News

2026-03-20

Let me share a quick summary and update on misalignment between LLM reasoning and manually annotated relations in mass media-news.

Almost a month ago we discovery the recent advanced in LLM reasoning. All the discussed and analyzed materials were formed into presentation with following slides (in Russian):

✨ 📰 Slides: https://nicolay-r.github.io/website/data/report_llm2023-nerel.pdf

Here is the three keypoint you might take out of it:

💚 1. High level of agreement. Reasoning within context for time-based relations. They are very grammar dependent, which it becomes the may reason model has a huge alignment rate. (Credits to transformer architecture and self-attention)

💛 2. Medum level of agreement. Locations and position related types of relations. LLM models may end up with non-definitive meaning of relation types. The exceptional cases are: relations that involve well-known politicians and countries.

❤️ 3. Low level of agreement were mainly caused by (i) terminology misalignment and (ii) sharper reasoning from LLM. This is a case where context is expected to be enriched and/or the meaning of relation type is better described.

For everything mentioned above, we eleminate such cases as: (i) issues with translation, (ii) network hallucination. If the first was mostly a technical trait of implementation, the hallucination is one thing that still remains poorly covered.

And thanks for AREkit double-s (AREkit-ss) which makes these studies available to replicate and spread on other languages. Feel free to check out or share them below to sample data with the one line! 🔥

💻 https://github.com/nicolay-r/arekit-ss/issues/52

Thank you for reading, interest and support! 🙏

🌟 AREkit “double-s”: https://github.com/nicolay-r/arekit-ss

🌟 AREkit core: https://github.com/nicolay-r/AREkit

How well ChatGPT-3.5 is Aligned With Manually Annotated Relations in Mass-Media News

2026-03-20

The recent LLM capabilities of understanding text relations are insane 🔥 Especially when it is well and nicely visualized!

I’ve tried to checkout how ChatGPT-3.5 is aligned with the manually annotated data, and that is what makes me think different about prompts!

This Wednesday, will be delighted to share the results of the largest known collection of annotated news 📰 in Russian with more than 40 relation types: NEREL (https://aclanthology.org/2021.ranlp-1.100/)

Feel free to join the event for more details 🙌

🗺 Talk: https://www.dropbox.com/scl/fi/xpfxlvxbpswjsxejk69qp/llm-nerel-report.m4v?rlkey=ilng4i53oreh2i27fj6427s7y&dl=0

⏰ When: 28 August, 19:00 (GMT+3, Moscow)

Presentation Language: Russian 🇷🇺

Powered by:

🌟 Project: https://github.com/nicolay-r/arekit-ss

🌟 Library: https://github.com/nicolay-r/AREkit

AREkit and What's new in Release 0.22.1

2026-03-20

We are happy to announce the AREkit version 21.1.

In this post we cover the most imporant changes were done since the prior release. The complete and large list of the updates could be found on the release page.

In short we first provide greater support of the BRAT-based annotation in order to adopt it in your own collections in order to use them for attitudes annotation and document sampling for training your machine learning. We proceed to develop a toolset and treat it as a preprocessor for OpenNRE framework. OpenNRE allows you to quickly develop a model adopted for the relation extraction specific problem, by being based on the JSON-like prepared data in terms of training/fine-tunning/evaluation stages. For now, AREkit allows us to do the preprocessing work with thespecific writer.

Alongside with the AREkit, we also update the ARElight-21.1. The latter allows you to perform the quick stabilization of the input text from your console. In terms of the list of the complete changes, please follow the release notes.

And finally we have a nice tutorials journey! (see image below) This is a list of examples over the sequence of the three major points of the application. We collect all the tutorials and separate them onto the three stage, related to:

embedding the custom collection;
declaration of the pipelines for
- text processing;
- attitudes annotation;
contexts with text opinions serialization.

And all of the mentioned above does not cover all the features added so far. The complete and large list of the updates you can find on the release page in greater details.

Thank you for reading!

Setup JAX framework with GPU support

2026-03-20

Calculation of the derivatives plays a significant role in neural networks tuning. The computational effectivenes is very crusial considering a large models and ability to train them. Besides the CPU and one of the most common option for calculations, recent advances finds a significant application of GPU and TPUs mostly because of a potentially greater performance vs. the central processing unit. Obvioulsy, such features won’t be available without a special software support, required for networks compilation towards the targeted processing unit.

JAX, rougly speaking, is a library, proposed and maintained by Google which combines autograd and XLA compiler in order to bring neural networks onto the computational devices in a most efficient way. However, speaking about GPU and more nn oriented devices as TPU, it is pretty important to perform a proper library setup in order to make them available to use. JAX is widely used for transformers, where T5 founds its implementation, including other variations formed into another flaxformer project.

In this post we address on the issue you may encountered with once decided to apply Jax library for GPU calculations. It is required to make library frienly and familiar with such toolkits: NVidia CUDA compiler (ncdu), CUDA DNN (cudnn). Most of the steps were taken from the JAX installation with Nvidia CUDA and cudNN support video by Avkash Chauhan.

Let’s get started!

The problem that you may enconter first is that you got installed the ordinary version of the related library dubbed as jaxlib. For example and in case of training T5 model, you may see the following logs:

[xla_bridge.py:356] Unable to initialize backend 'tpu_driver': NOT_FOUND: Unable to find driver in registry given worker:
[xla_bridge.py:356] Unable to initialize backend 'cuda': module 'jaxlib.xla_extension' has no attirbute 'GpuAllocatorConfig'
[xla_bridge.py:356] Unable to initialize backend 'rocm': module 'jaxlib.xla_extension' has no attirbute 'GpuAllocatorConfig'
...
[xla_bdridge.py:363] No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)

At first, let’s say we have installed:

jax==0.3.21
jaxlib==0.3.15

Here is list of actions required to be performed in order to make it familiar with GPU devices:

Uninstall jax and jaxlib in order to perform its clean installation later:
```
pip uninstall jax jaxlib
```

cudnn version checkout:

cat /usr/include/x86_64-linux-gnu/cudnn_v*.h | grep CUDNN

Clean install from list of laxlib-cudnn-cuda. At this point is pretty important to checkout the version of cuda and cudnn pre-installed and to be used by jaxlib; According to the list of the available pip packages we were down for jaxlib==0.3.15+cuda11.cudnn82 and therefore perform the installation as follows:
```
pip install --upgrade jax==0.3.15 jaxlib==0.3.15+cuda11.cudnn82 \
  -f https://storage.googleapis.com/jax-releases/cuda11/jaxlib-0.3.15+cuda11.cudnn82-cp39-none-manylinux2014_x86_64.whl
```

check out:

>>> import jax
>>> jax.devices()
[GpuDevice(id=0, process_index=0), ... GpuDevice(id=3, process_index=0)]

Reference

JAX installation with Nvidia CUDA and cudNN support by Avkash Chauhan

AREkit Tutorial: Sample Mass-Media Text Opinions for BERT

2026-03-20

In this tutorial we provide a list of steps required to prepare samples with text opinions for BERT language model.

Complete tutorial code implementation

NOTE: This post represents an updated version of the prior one “Process Mass-Media relations for Language Models with AREkit” ; in the prior one we describe sampling process from scratch and under older API version AREkit-0.22.0.

Sampler Initialization

First, it is necessary to declare labels expected to adopted in further samples preparation process. In this post we focused on sentiment-related data sampling and therefore considering the following set of labels: Positive, Negative and additionally neutral, type of NoLabel which AREkit provides by default.

class Positive(Label):
    pass

class Negative(Label):
    pass

Next step, we declare label scaler. Scaler (BaseLabelScaler class) allows us to provide conversion from Label type to int/uint values and vice versa. We declare Sentiment scaller as follows:

class SentimentLabelScaler(BaseLabelScaler):
    def __init__(self):
        int_to_label = OrderedDict([(NoLabel(), 0), (Positive(), 1), (Negative(), -1)])
        uint_to_label = OrderedDict([(NoLabel(), 0), (Positive(), 1), (Negative(), 2)])
        super(SentimentLabelScaler, self).__init__(int_to_label, uint_to_label)

In terms of the input aspects of the BERT model, we deal with a sequence (optionally) separated by a [SEP] token onto couple parts, such as: TextA and TextB. For the classificational task, TextB might be treated as a prompt with the auxilary information which might be considered in a result class decission.

For the sentiment analysis and relation extraction domain you may examine more approaches in Awesome Sentiment Attitude Extraction Repository

At present, text_b template is expected to contain a placeholders for subject, object and context, where context corresponds to a text part between subject and object. For texts in Russian, we assign the following NLI-styled (Natural Language Inference) prompt:

NOTE: you may left text_b_tempalete as None once you don’t want to consider a separated sequence.

text_b_template = '{subject} к {object} в контексте : << {context} >>'

Next, we focused on text provider. First, there is a need to setup terms mapper. Terms mappers allows us to customize the way on how terms will be displayed in samples. AREkit provides BertDefaultStringTextTermsMapper, in which you may among all of the different term types customize mentioned named entities.

In terms of the latter we have a separated post AREkit Tutorial: Entity Values Formatting Examples. From that tutorial, here we adopt CustomEntitiesFormatter and assign #S and #O masks towards the text opinion participants, i.e. subject and object respectively.

Depending on the text_b_template we may declare a single text provider (i.e. TextA only) or pair-based one:

terms_mapper = BertDefaultStringTextTermsMapper(
    entity_formatter=CustomEntitiesFormatter(
        subject_fmt="#S", object_fmt="#O"))

text_provider = BaseSingleTextProvider(terms_mapper) \
    if text_b_template is None else \
        PairTextProvider(text_b_template, terms_mapper)

Finally we may compose sample rows provider:

sample_rows_provider = BaseSampleRowProvider(
    label_provider=MultipleLabelProvider(SentimentLabelScaler()),
    text_provider=text_provider)

Initialize information related to the samples format and output directory/path. As for format, there is a need to declare a type inherited from the BaseWriter. By default, AREkit provides TsvWriter – is a CSV-style formatter.

Side note: Tilte prefix tsv comes from the format proposed by google-BERT.

writer = TsvWriter(write_header=True)
samples_io = SamplesIO("out/", writer, target_extension=".tsv.gz")

pipeline_item = BertExperimentInputSerializerPipelineItem(
    sample_rows_provider=sample_rows_provider,
    samples_io=samples_io,
    save_labels_func=lambda data_type: True,
    balance_func=lambda data_type: data_type == DataType.Train)

Running Sampler

Complete tutorial code implementation

Please refer to the following posts in order to initialize your text opinion annotation pipeline (annot_pipeline) and setup Data Folding (data_folding):

Or just follow the complete tutorial implemenation

Finally, we can compose pipeline by wrapping a predefined pipeline_item and then run it! This could be accomplished as follows:

pipeline = BasePipeline([
    pipeline_item
])

pipeline.run(input_data=None,
             params_dict={
                 "data_folding": data_folding,
                 "data_type_pipelines": annot_pipeline 
             })

Finally our result is a content of the out directory. The contents depend on Data Folding format. For example, in case of the fixed folding onto Train and Test data types, it is expected to see the following set of contents:

./out/
    sample_train.tsv.gz
    sample_test.tsv.gz

AREkit Tutorial: Entity Values Formatting Examples

2026-03-20

This short post illustrates implementation of the base entity formatter. Entity formatter is requred for formatting values of mentioned named entities, masking them.

AREkit proposes an internal type OpinionEntityType, which describes all the possible types that opinion participants (subject/object) might be. This enumeration type includes the following types:

class OpinionEntityType(Enum):
    Object = 1
    Subject = 2
    SynonymSubject = 3
    SynonymObject = 4
    Other = 5

Implementation of the base class of the entities formatting illustrated in a snippet below:

class StringEntitiesFormatter(object):
    def to_string(self, original_value, entity_type):
        raise NotImplementedError()

Inherited versions of the base class illustrated below:

class CustomEntitiesFormatter(StringEntitiesFormatter):

    def __init__(self, subject_fmt="[subject]", object_fmt="[object]"):
        self.__subj_fmt = subject_fmt
        self.__obj_fmt = object_fmt

    def to_string(self, original_value, entity_type):
        assert(isinstance(original_value, Entity))
        if entity_type == OpinionEntityType.Other:
            return original_value.Value
        elif entity_type == OpinionEntityType.Object or
                entity_type == OpinionEntityType.SynonymObject:
            return self.__obj_fmt
        elif entity_type == OpinionEntityType.Subject or
                entity_type == OpinionEntityType.SynonymSubject:
            return self.__subj_fmt 
        return None

List of the other entity formatters provided out-of-the-box is as follows:

RussianEntitiesCasedFormatter – supports russian cases for subject (субъект) and object (объект).

NOTE It was decided not to mention other adapters since they are related to CustomEntitiesFormatter described above.