Skip to content

Package Documentation

Scraped Lineups API Package.

Classes:

Name Description
ScrapedLineupsAPI

Represents Signal's Scraped Lineups API.
ScrapedLineup

Scraped Lineup.

ScrapedLineup dataclass

Detailed information about a scraped lineup.

Attributes:

Name Type Description
lineup_id int

Integer. A unique identifier of the lineup line.
message_id Optional[int]

Integer. A unique identifier of the message containing the specific lineup. A message can contain more than one lineup.
external_message_id Optional[str]

String. It serves as a unique identifier for a message, supplied by any company that has integrated with Signal.
parsed_part_id Optional[int]

Integer. A unique identifier for each email part. The email body and each attachment are considered different parsed parts. For an example the email body and its pdf attachment have same MessageID and different ParsedPartID.
line_from Optional[int]

Nullable integer. The starting line from which the lineup is extracted. The email subject counts as line 0 and the body content starts from line 1.
line_to Optional[int]

Nullable integer. The final line from which the lineup is extracted. For single line lineups LineFrom is equal to LineTo. For descriptive lineups that span across multiple lines we have LineTo>LineFrom. These two numbers help the user identify which part of the text has been used to extract the lineup data.
in_line_order Optional[int]

Nullable integer. This integer is used to list different cargoes extracted from the same line. It is the case for lineups with multiple cargoes or buyers. A lineup with two cargoes like 'gasoil + fuel oil' is interpreted in our system as 2 lineups with same MessageID, same ParsedPartID, same LineNumber, same IMO, different LineupID and an incremental InLineOrder number.
source Optional[str]

String. It describes the source of the information. Our system allows the user to inject data in many different ways, namely through email (Source='Email'), through Slack channels (Source='Slack') or through manual contributions directly from our frontend platform TSOP (Source='User').
updated_date Optional[datetime]

String, format YYYY-MM-DD HH:MM:SS, UTC timezone. Date on which the lineup has been reevaluated for the last time. In case of an email received by a broker one month ago and reprocessed through our engine today, this date will be today's.
received_date Optional[datetime]

String, format YYYY-MM-DD HH:MM:SS, UTC timezone. Date on which the lineup has been injected into our system and processed.
is_deleted Optional[bool]

Boolean. This value is true if the lineup is marked as Deleted.
scraped_vessel_name Optional[str]

String. The vessel name as reported in the lineup line, i.e. 'Signal Alpha', 'Cpt A Stellatos', 'Genco Tiberius'. 'TBN' can also be found.
scraped_imo Optional[str]

String. Vessel's IMO as reported in the lineup line, i.e. '9412036', '9439670', '9331555'.
scraped_deadweight Optional[str]

String. The dead weight of the vessel as reported in the lineup line, i.e. '150249', '167'.
scraped_year_built Optional[str]

String. The year built of the vessel as reported in the lineup line, i.e. '2004', '09'.
imo Optional[int]

Integer. The seven-digits number that uniquely identifies the ship reported in the lineup. It is the result of our internally developed Vessel Mapper model.
vessel_name Optional[str]

String. It is the current vessel name corresponding to the IMO mentioned above. Provided to better specify the vessel and its particulars. Source: our internal Vessel Database.
deadweight Optional[int]

Integer. The dead weight in tonnes [t] corresponding to the IMO mentioned above.
year_built Optional[int]

Integer, YYYY format. The year the vessel was built. Source: our internal Vessel Database.
liquid_capacity Optional[int]

Integer, measured in cbm [cbm]. The liquid capacity of the IMO mentioned above. Source: our internal Vessel Database.
vessel_type_id Optional[int]

Integer. Numeric ID corresponding to the different values of the VesselType field. 1-> Tanker, 3-> Dry, 4 -> Containers, 5 ->LNG (Liquified Natural gas), 6-> LPG (Liquified Petroleum Gas).
vessel_type Optional[str]

String. Description of the type of the vessel (VesselTypeID), based on the carried cargo. Main categories are Tankers, Dry (bulk carriers), Containers, LNG and LPG.
vessel_class_id Optional[int]

Integer. It is an ID corresponding to the different vessel classes of a certain vessel type, as split according to our internal Vessel Database. For example 84->VLCC, 85->Suezmax, 70->Capesize.
vessel_class Optional[str]

String. Name of the vessel class the vessel belongs to. Assignment of a vessel to a certain VesselClass is based on the VesselType and the value of its Deadweight (if Tanker or Dry), its LiquidCap (if LNG/LPG) or its TEU (if Containers). For example, an Aframax is a Tanker vessel with Deadweight within the range 82kt - 125kt, while a Capesize is a Dry vessel with Deadweight within the range 120kt-220kt. LR2 are defined as Aframax, as only Deadweight is used to define vessel classes.
commercial_operator_id Optional[int]

Integer. Numeric ID corresponding to the current maritime company that commercially manages the vessel corresponding to the IMO mentioned above. Source: Signal's proprietary algorithm.
commercial_operator Optional[str]

String. Name of the current maritime company associated to CommercialOperatorID. Source: our internal Companies Database.
scraped_eta Optional[str]

String. The estimated time of arrival of the lineup as reported in the original text.
eta Optional[datetime]

Date, format YYYY-MM-DD. The mapped date corresponding to the estimated time of arrival date.
scraped_etb Optional[str]

String. The estimated time of berthing of the lineup as reported in the original text. Strings tba (to be announced) and tbc (to be confirmed) can also be found.
etb Optional[datetime]

Date, format YYYY-MM-DD. The mapped date corresponding to the estimated time of berthing date.
scraped_etd Optional[str]

String. The estimated time of departure of the lineup as reported in the original text. Strings tba (to be announced) and tbc (to be confirmed) can also be found.
etd Optional[datetime]

Date, format YYYY-MM-DD. The mapped date corresponding to the estimated time of departure date.
scraped_location Optional[str]

String. The location reported in the original text of the lineup. Also, It could be reported in the subject of email.
location_geo_id Optional[int]

Integer. It is the internal ID of the mapped location reported in the lineup. Our models convert and map a scraped string containing geo information to a specific entity of the Signal's proprietary geofencing structure. Each geo entity is identified by a taxonomy, an ID and a name. Examples: the string 'turkmenbashi' is mapped to a geo asset with name 'Turkmenbashi Refinery', ID 5086 and taxonomy GoeAsset (TaxonomyID=1); the string 'murmansk' is mapped to a geoasset with name 'Murmansk', ID 3761 and taxonomy Port (TaxonomyID=2); the string 'congo' is mapped to a geoasset with name 'Congo', ID 49 and taxonomy Country (TaxonomyID=3).
location_name Optional[str]

String. The name of the Signal geo entity related to the reported location of the lineup. Examples: 'Qingdao', 'Nigeria', 'Yangpu', 'Quanzhou'.
location_taxonomy_id Optional[int]

Integer. An internal ID corresponding to each taxonomy, the level of geo details, from 1 to 7. A terminal (geoasset) has the lowest taxonomy (TaxonomyID=1), a port has TaxonomyID=2, while countries and wider areas have higher taxonomy (TaxonomyID>=3). Examples of Level 0 areas (TaxonomyID=4) include 'Arabian Gulf', 'US Gulf' and 'East Mediterranean'. Level 1 areas (TaxonomyID=5) consist of multiple level 0 areas (TaxonomyID=4). For example, level 1 area 'Mediterranean' groups together the level 0 areas 'West Mediterranean', 'Central Mediterranean' and 'East Mediterranean'. Level 2 areas (TaxonomyID=6) consist of multiple level 1 areas (TaxonomyID=4). For example, level 2 area 'Mediterranean/UK Continent' groups together the 'Mediterranean' and 'UK Continent' level 1 areas. Level 3 areas (TaxonomyID=7) are the highest area grouping in our taxonomy and consist of multiple level 2 areas (TaxonomyID=6). Examples of such areas are 'Pacific America' or 'Africa'. These group together level 2 areas. For instance, 'Pacific America' groups together the level 2 areas West Coast North America', 'West Coast Mexico', 'West Coast Central America' and 'West Coast South America'.
location_taxonomy Optional[str]

String. The extended name identifying the TaxonomyID. Possible values are: GeoAsset-> 1, Port -> 2, Country-> 3, Level0->4, Level1->5, Level2->6, Level3->7.
operation_type_id Optional[int]

Interger. Numeric ID corresponding to the different values of the OperationType field. 1-> Loading, 2-> Discharge, 3 -> Waypoint, -2 -> NotSet, -1 -> Unknown.
operation_type Optional[str]

String. Denotes the operation type of the lineup if explicitly mentioned, like 'loading' for load.
scraped_quantity Optional[str]

String. The quantity as reported in the original text of the lineup.
quantity Optional[float]

Integer. The mapped quantity measured in the corresponding value of QuantityUnit filed.
quantity_unit Optional[str]

String. Denotes the quantity unit of the lineup. 1-> Tonnes, 2-> Barrels, -2 -> NotSet, -1 -> Unknown.
scraped_cargo_type Optional[str]

String. The cargo type of the cargo as reported in the original text, often shortened. Examples: 'nhc', 'ulsd', 'ums', 'nap', 'go'.
cargo_type_id Optional[int]

Integer. It is an internal ID corresponding to the mapped cargo type of the cargo. A proprietary model is responsible to match the reported cargo string to a specific cargo type in our hierarchy. Examples: 19-> Crude Oil, 16->Fueloil, 9-> Naphtha, 135-> Unleaded Motor Spirit, 12-> Gasoil, 60-> 'Ultra Low Sulphur Diesel (ULSD 10ppm)'.
cargo_type Optional[str]

String. The extended name corresponding to the CargoTypeID field. Source: our internal CargoTypes database.
cargo_group_id Optional[int]

Integer. Numeric ID corresponding to the high-level cargo type of the reported cargo type. It is provided to group cargoes and facilitate analytics. Examples: 130000->Dirty, 120000->Clean, 110000->IMO.
cargo_group Optional[str]

String. The extended name corresponding to the CargoGroupID field. Source: our internal CargoTypes database.
scraped_api_gravity Optional[str]

String. The API gravity reported in the original text on the lineup.
api_gravity Optional[int]

Numeric. The mapped value corresponding to the API gravity.
scraped_origin Optional[str]

String. The origin location reported in the original text of the lineup.
origin_geo_id Optional[int]

Integer. An internal ID corresponding to the mapped origin location of the lineup. See LocationGeoID for more details.
origin_name Optional[str]

String. The name of the Signal geo entity related to the reported origin location of the lineup. Examples: 'Fuel Pier', 'Port Of Arzew Terminal 2', 'CPC (Novorossiysk) SBM', 'Sheskharis Oil Terminal'.
origin_taxonomy_id Optional[int]

Integer. An internal ID corresponding to the taxonomy of the mapped origin location. See LocationTaxonomyID for more details.
origin_taxonomy Optional[str]

String. The extended name identifying the TaxonomyID. Possible values are: GeoAsset-> 1, Port -> 2, Country-> 3, Level0->4, Level1->5, Level2->6, Level3->7.
scraped_destination Optional[str]

String. The destination location reported in the original text of the lineup. Strings tba (to be announced) and tbn (to be nominated) can also be found
destination_geo_id Optional[int]

Integer. An internal ID corresponding to the mapped destination location of the lineup. See LocationGeoID for more details.
destination_name Optional[str]

String. The name of the Signal geo entity related to the reported destination location of the lineup. Examples: 'Fuel Pier', 'Port Of Arzew Terminal 2', 'CPC (Novorossiysk) SBM', 'Sheskharis Oil Terminal'.
destination_taxonomy_id Optional[int]

Integer. An internal ID corresponding to the taxonomy of the mapped destination location. See LocationTaxonomyID for more details.
destination_taxonomy Optional[str]

String. The extended name identifying the TaxonomyID. Possible values are: GeoAsset-> 1, Port -> 2, Country-> 3, Level0->4, Level1->5, Level2->6, Level3->7.
scraped_supplier Optional[str]

String. The supplier as reported in the original text of the lineup.
supplier_id Optional[int]

Integer. Numeric ID corresponding to the supplier company that it is reported in the line. We use an internal mapper to find the correspondence between the reported string and our database.
supplier Optional[str]

String. The company name corresponding to the SupplierID field. Provided to better specify the company involved in the business. Source: our internal Company Database.
scraped_charterer Optional[str]

String. The charterer as reported in the original text of the lineup.
charterer_id Optional[int]

Integer. Numeric ID corresponding to the chartering company that it is reported in the line. We use an internal mapper to find the correspondence between the reported string and our database.
charterer Optional[str]

String. The company name corresponding to the ChartererID field. Provided to better specify the company involved in the business. Source: our internal Company Database.
scraped_buyer Optional[str]

String. The buyer as reported in the original text of the lineup.
buyer_id Optional[int]

Integer. Numeric ID corresponding to the buyer company that it is reported in the line. We use an internal mapper to find the correspondence between the reported string and our database.
buyer Optional[str]

String. The company name corresponding to the BuyerID field. Provided to better specify the company involved in the business. Source: our internal Company Database.
scraped_port_agent Optional[str]

String. The port agent as reported in the original text of the lineup.
port_agent_id Optional[int]

Integer. Numeric ID corresponding to the port agent company that it is reported in the line.
port_agent Optional[str]

String. The company name corresponding to the PortAgentID field. Provided to better specify the company involved in the business. Source: our internal Company Database.
vessel_status_id Optional[int]

Integer. Numeric ID corresponding to the different values of the VesselStatus field. 1-> Expected, 2-> Arrived, 3-> At anchor, 4-> At berth, 5-> Sailed, 6-> Failed, 7-> Cancelled, 8-> Substituted, -2 -> NotSet, -1 -> Unknown.
vessel_status Optional[str]

String denoting the vessel status of a lineup if explicitly mentioned, like 'expected for expected or 'waiting' for at anchor.
content Optional[str]

String. The full content of the lineup. For a single line lineup it is the line content. For multi line lineups it is the collection of all the relevant parts of the text.
subject Optional[str]

String. The email subject of the lineup. This field has content when Source='Email'.
sender Optional[str]

String. Our own mapping of the shipping company sending out the market report through email. This string helps grouping emails sent by the same organization, but from different domains. It is often the case for big organizations operating worldwide. For example Sender= 'SSY' for both domains 'ssysin.com' and 'ssy.co'.
is_private Optional[bool]

Boolean. A lineup is private if injected by a user into his own private account within TSOP. A user can provide private information through email forwarding or through Slack. Private lineup information stay in the account, are accessible by the account users only (people within the same company) and are the most valuable ones.
Source code in signal_ocean/scraped_lineups/models.py 
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
@dataclass(frozen=True)
class ScrapedLineup:
    """Detailed information about a scraped lineup.

    Attributes:
        lineup_id: Integer. A unique identifier of the lineup line.
        message_id: Integer. A unique identifier of the message containing the
            specific lineup. A message can contain more than one lineup.
        external_message_id: String. It serves as a unique identifier for a
            message, supplied by any company that has integrated with Signal.
        parsed_part_id: Integer. A unique identifier for each email part. The
            email body and each attachment are considered different parsed
            parts. For an example the email body and its pdf attachment have
            same MessageID and different ParsedPartID.
        line_from: Nullable integer. The starting line from which the lineup
            is extracted. The email subject counts as line 0 and the body
            content starts from line 1.
        line_to: Nullable integer. The final line from which the lineup is
            extracted. For single line lineups LineFrom is equal to LineTo.
            For descriptive lineups that span across multiple lines we have
            LineTo>LineFrom. These two numbers help the user identify which
            part of the text has been used to extract the lineup data.
        in_line_order: Nullable integer. This integer is used to list different
            cargoes extracted from the same line. It is the case for lineups
            with multiple cargoes or buyers. A lineup with two cargoes like
            'gasoil + fuel oil' is interpreted in our system as 2 lineups with
            same MessageID, same ParsedPartID, same LineNumber,  same IMO,
            different LineupID and an incremental InLineOrder number.
        source: String. It describes the source of the information. Our system
            allows the user to inject data in many different ways, namely
            through email (Source='Email'), through Slack channels
            (Source='Slack') or through manual contributions directly from
            our frontend platform TSOP (Source='User').
        updated_date: String, format YYYY-MM-DD HH:MM:SS, UTC timezone.
            Date on which the lineup has been reevaluated for the last time.
            In case of an email received by a broker one month ago and
            reprocessed through our engine today, this date will be today's.
        received_date: String, format YYYY-MM-DD HH:MM:SS, UTC timezone.
            Date on which the lineup has been injected into our system
            and processed.
        is_deleted: Boolean. This value is true if the lineup is marked as
            Deleted.
        scraped_vessel_name: String. The vessel name as reported in the lineup
            line, i.e. 'Signal Alpha', 'Cpt A Stellatos', 'Genco Tiberius'.
            'TBN' can also be found.
        scraped_imo: String. Vessel's IMO as reported in the lineup line,
            i.e. '9412036', '9439670', '9331555'.
        scraped_deadweight: String. The dead weight of the vessel as reported
            in the lineup line, i.e. '150249', '167'.
        scraped_year_built: String. The year built of the vessel as reported
            in the lineup line, i.e. '2004', '09'.
        imo: Integer. The seven-digits number that uniquely identifies the
            ship reported in the lineup. It is the result of our internally
            developed Vessel Mapper model.
        vessel_name: String. It is the current vessel name corresponding to
            the IMO mentioned above. Provided to better specify the vessel and
            its particulars. Source: our internal Vessel Database.
        deadweight: Integer. The dead weight in tonnes [t] corresponding to
            the IMO mentioned above.
        year_built: Integer, YYYY format. The year the vessel was built.
            Source: our internal Vessel Database.
        liquid_capacity: Integer, measured in cbm [cbm]. The liquid capacity
            of the IMO mentioned above. Source: our internal Vessel Database.
        vessel_type_id: Integer. Numeric ID corresponding to the different
            values of the VesselType field. 1-> Tanker, 3-> Dry,
            4 -> Containers, 5 ->LNG (Liquified Natural gas),
            6-> LPG (Liquified Petroleum Gas).
        vessel_type: String. Description of the type of the vessel
            (VesselTypeID), based on the carried cargo. Main categories are
            Tankers, Dry (bulk carriers), Containers, LNG and LPG.
        vessel_class_id: Integer. It is an ID corresponding to the different
            vessel classes of a certain vessel type, as split according
            to our internal Vessel Database.
            For example 84->VLCC, 85->Suezmax, 70->Capesize.
        vessel_class: String. Name of the vessel class the vessel belongs to.
            Assignment of a vessel to a certain VesselClass is based on the
            VesselType and the value of its Deadweight (if Tanker or Dry),
            its LiquidCap (if LNG/LPG) or its TEU (if Containers).
            For example, an Aframax is a Tanker vessel with Deadweight within
            the range 82kt - 125kt, while a Capesize is a Dry vessel with
            Deadweight within the range 120kt-220kt. LR2 are defined
            as Aframax, as only Deadweight is used to define vessel classes.
        commercial_operator_id: Integer. Numeric ID corresponding to the
            current maritime company that commercially manages the vessel
            corresponding to the IMO mentioned above.
            Source: Signal's proprietary algorithm.
        commercial_operator: String. Name of the current maritime company
            associated to CommercialOperatorID.
            Source: our internal Companies Database.
        scraped_eta: String. The estimated time of arrival of the lineup as
            reported in the original text.
        eta: Date, format YYYY-MM-DD. The mapped date corresponding to the
            estimated time of arrival date.
        scraped_etb: String. The estimated time of berthing of the lineup as
            reported in the original text. Strings tba (to be announced) and
            tbc (to be confirmed) can also be found.
        etb: Date, format YYYY-MM-DD. The mapped date corresponding to the
            estimated time of berthing date.
        scraped_etd: String. The estimated time of departure of the lineup as
            reported in the original text. Strings tba (to be announced) and
            tbc (to be confirmed) can also be found.
        etd: Date, format YYYY-MM-DD. The mapped date corresponding to the
            estimated time of departure date.
        scraped_location: String. The location reported in the original text
            of the lineup. Also, It could be reported in the subject of email.
        location_geo_id: Integer. It is the internal ID of the mapped location
            reported in the lineup. Our models convert and map a scraped string
            containing geo information to a specific entity of the Signal's
            proprietary geofencing structure. Each geo entity is identified by
            a taxonomy, an ID and a name. Examples: the string 'turkmenbashi'
            is mapped to a geo asset with name 'Turkmenbashi Refinery', ID
            5086 and taxonomy GoeAsset (TaxonomyID=1); the string 'murmansk' is
            mapped to a geoasset with name 'Murmansk', ID 3761 and taxonomy
            Port (TaxonomyID=2); the string 'congo' is mapped to a geoasset
            with name 'Congo', ID 49 and taxonomy Country (TaxonomyID=3).
        location_name: String. The name of the Signal geo entity related to
            the reported location of the lineup. Examples: 'Qingdao',
            'Nigeria', 'Yangpu', 'Quanzhou'.
        location_taxonomy_id: Integer. An internal ID corresponding to each
            taxonomy, the level of geo details, from 1 to 7. A terminal
            (geoasset) has the lowest taxonomy (TaxonomyID=1), a port has
            TaxonomyID=2, while countries and wider areas have higher taxonomy
            (TaxonomyID>=3). Examples of Level 0 areas (TaxonomyID=4) include
            'Arabian Gulf', 'US Gulf' and 'East Mediterranean'. Level 1 areas
            (TaxonomyID=5) consist of multiple level 0 areas (TaxonomyID=4).
            For example, level 1 area 'Mediterranean' groups together the
            level 0 areas 'West Mediterranean', 'Central Mediterranean' and
            'East Mediterranean'. Level 2 areas (TaxonomyID=6) consist of
            multiple level 1 areas (TaxonomyID=4). For example, level 2 area
            'Mediterranean/UK Continent' groups together the 'Mediterranean'
            and 'UK Continent' level 1 areas. Level 3 areas (TaxonomyID=7) are
            the highest area grouping in our taxonomy and consist of multiple
            level 2 areas (TaxonomyID=6). Examples of such areas are
            'Pacific America' or 'Africa'. These group together level 2 areas.
            For instance, 'Pacific America' groups together the level 2 areas
             West Coast North America', 'West Coast Mexico',
            'West Coast Central America' and 'West Coast South America'.
        location_taxonomy: String. The extended name identifying the
            TaxonomyID. Possible values are: GeoAsset-> 1, Port -> 2,
            Country-> 3, Level0->4, Level1->5, Level2->6, Level3->7.
        operation_type_id: Interger. Numeric ID corresponding to the different
            values of the OperationType field. 1-> Loading, 2-> Discharge,
            3 -> Waypoint, -2 -> NotSet, -1 -> Unknown.
        operation_type: String. Denotes the operation type of the lineup if
            explicitly mentioned, like 'loading' for load.
        scraped_quantity: String. The quantity as reported in the original
            text of the lineup.
        quantity: Integer. The mapped quantity measured in the corresponding
            value of QuantityUnit filed.
        quantity_unit: String. Denotes the quantity unit of the lineup.
            1-> Tonnes, 2-> Barrels, -2 -> NotSet, -1 -> Unknown.
        scraped_cargo_type: String. The cargo type of the cargo as reported in
            the original text, often shortened. Examples: 'nhc', 'ulsd',
            'ums', 'nap', 'go'.
        cargo_type_id: Integer. It is an internal ID corresponding to the
            mapped cargo type of the cargo. A proprietary model is responsible
            to match the reported cargo string to a specific cargo type in our
            hierarchy. Examples: 19-> Crude Oil, 16->Fueloil, 9-> Naphtha,
            135-> Unleaded Motor Spirit, 12-> Gasoil,
            60-> 'Ultra Low Sulphur Diesel (ULSD 10ppm)'.
        cargo_type: String. The extended name corresponding to the CargoTypeID
            field. Source: our internal CargoTypes database.
        cargo_group_id: Integer. Numeric ID corresponding to the high-level
            cargo type of the reported cargo type. It is provided to group
            cargoes and facilitate analytics. Examples: 130000->Dirty,
            120000->Clean, 110000->IMO.
        cargo_group: String. The extended name corresponding to the
            CargoGroupID field. Source: our internal CargoTypes database.
        scraped_api_gravity: String. The API gravity reported in the original
            text on the lineup.
        api_gravity: Numeric. The mapped value corresponding to the API
            gravity.
        scraped_origin: String. The origin location reported in the original
            text of the lineup.
        origin_geo_id: Integer. An internal ID corresponding to the mapped
            origin location of the lineup. See LocationGeoID for more details.
        origin_name: String. The name of the Signal geo entity related to the
            reported origin location of the lineup. Examples: 'Fuel Pier',
            'Port Of Arzew Terminal 2', 'CPC (Novorossiysk) SBM',
            'Sheskharis Oil Terminal'.
        origin_taxonomy_id: Integer. An internal ID corresponding to the
            taxonomy of the mapped origin location. See LocationTaxonomyID
            for more details.
        origin_taxonomy: String. The extended name identifying the TaxonomyID.
            Possible values are: GeoAsset-> 1, Port -> 2, Country-> 3,
            Level0->4, Level1->5, Level2->6, Level3->7.
        scraped_destination: String. The destination location reported in the
            original text of the lineup. Strings tba (to be announced) and tbn
            (to be nominated) can also be found
        destination_geo_id: Integer. An internal ID corresponding to the
            mapped destination location of the lineup. See LocationGeoID for
            more details.
        destination_name: String. The name of the Signal geo entity related to
            the reported destination location of the lineup. Examples:
            'Fuel Pier', 'Port Of Arzew Terminal 2',
            'CPC (Novorossiysk) SBM', 'Sheskharis Oil Terminal'.
        destination_taxonomy_id: Integer. An internal ID corresponding to the
            taxonomy of the mapped destination location.
            See LocationTaxonomyID for more details.
        destination_taxonomy: String. The extended name identifying the
            TaxonomyID. Possible values are: GeoAsset-> 1, Port -> 2,
            Country-> 3, Level0->4, Level1->5, Level2->6, Level3->7.
        scraped_supplier: String. The supplier as reported in the original
            text of the lineup.
        supplier_id: Integer. Numeric ID corresponding to the supplier company
            that it is reported in the line. We use an internal mapper to find
            the correspondence between the reported string and our database.
        supplier: String. The company name corresponding to the SupplierID
            field. Provided to better specify the company involved in the
            business. Source: our internal Company Database.
        scraped_charterer: String. The charterer as reported in the original
            text of the lineup.
        charterer_id: Integer. Numeric ID corresponding to the chartering
            company that it is reported in the line. We use an internal mapper
            to find the correspondence between the reported string and our
            database.
        charterer: String. The company name corresponding to the ChartererID
            field. Provided to better specify the company involved in the
            business. Source: our internal Company Database.
        scraped_buyer: String. The buyer as reported in the original text of
            the lineup.
        buyer_id: Integer. Numeric ID corresponding to the buyer company that
            it is reported in the line. We use an internal mapper to find the
            correspondence between the reported string and our database.
        buyer: String. The company name corresponding to the BuyerID field.
            Provided to better specify the company involved in the business.
            Source: our internal Company Database.
        scraped_port_agent: String. The port agent as reported in the original
            text of the lineup.
        port_agent_id: Integer. Numeric ID corresponding to the port agent
            company that it is reported in the line.
        port_agent: String. The company name corresponding to the PortAgentID
            field. Provided to better specify the company involved in the
            business. Source: our internal Company Database.
        vessel_status_id: Integer. Numeric ID corresponding to the different
            values of the VesselStatus field. 1-> Expected, 2-> Arrived,
            3-> At anchor, 4-> At berth, 5-> Sailed, 6-> Failed,
            7-> Cancelled, 8-> Substituted, -2 -> NotSet, -1 -> Unknown.
        vessel_status: String denoting the vessel status of a lineup if
            explicitly mentioned, like 'expected for expected or 'waiting' for
            at anchor.
        content: String. The full content of the lineup. For a single line
            lineup it is the line content. For multi line lineups it is the
            collection of all the relevant parts of the text.
        subject: String. The email subject of the lineup. This field has
            content when Source='Email'.
        sender: String. Our own mapping of the shipping company sending out the
            market report through email. This string helps grouping emails
            sent by the same organization, but from different domains. It is
            often the case for big organizations operating worldwide. For
            example Sender= 'SSY' for both domains 'ssysin.com' and 'ssy.co'.
        is_private: Boolean. A lineup is private if injected by a user into
            his own private account within TSOP. A user can provide private
            information through email forwarding or through Slack.
            Private lineup information stay in the account, are accessible by
            the account users only (people within the same company) and are
            the most valuable ones.
    """

    # entity details
    lineup_id: int
    message_id: Optional[int] = None
    external_message_id: Optional[str] = None
    parsed_part_id: Optional[int] = None
    line_from: Optional[int] = None
    line_to: Optional[int] = None
    in_line_order: Optional[int] = None
    source: Optional[str] = None
    updated_date: Optional[datetime] = None
    received_date: Optional[datetime] = None
    is_deleted: Optional[bool] = False

    # vessel
    scraped_vessel_name: Optional[str] = None
    scraped_imo: Optional[str] = None
    scraped_deadweight: Optional[str] = None
    scraped_year_built: Optional[str] = None
    imo: Optional[int] = None
    vessel_name: Optional[str] = None
    deadweight: Optional[int] = None
    year_built: Optional[int] = None
    liquid_capacity: Optional[int] = None
    vessel_type_id: Optional[int] = None
    vessel_type: Optional[str] = None
    vessel_class_id: Optional[int] = None
    vessel_class: Optional[str] = None
    commercial_operator_id: Optional[int] = None
    commercial_operator: Optional[str] = None

    # eta
    scraped_eta: Optional[str] = None
    eta: Optional[datetime] = None

    # etb
    scraped_etb: Optional[str] = None
    etb: Optional[datetime] = None

    # etd
    scraped_etd: Optional[str] = None
    etd: Optional[datetime] = None

    # location
    scraped_location: Optional[str] = None
    location_geo_id: Optional[int] = None
    location_name: Optional[str] = None
    location_taxonomy_id: Optional[int] = None
    location_taxonomy: Optional[str] = None

    # operation_type
    operation_type_id: Optional[int] = None
    operation_type: Optional[str] = None

    # quantity
    scraped_quantity: Optional[str] = None
    quantity: Optional[float] = None
    quantity_unit: Optional[str] = None

    # cargo type
    scraped_cargo_type: Optional[str] = None
    cargo_type_id: Optional[int] = None
    cargo_type: Optional[str] = None
    cargo_group_id: Optional[int] = None
    cargo_group: Optional[str] = None

    # api_gravity
    scraped_api_gravity: Optional[str] = None
    api_gravity: Optional[int] = None

    # origin
    scraped_origin: Optional[str] = None
    origin_geo_id: Optional[int] = None
    origin_name: Optional[str] = None
    origin_taxonomy_id: Optional[int] = None
    origin_taxonomy: Optional[str] = None

    # destination
    scraped_destination: Optional[str] = None
    destination_geo_id: Optional[int] = None
    destination_name: Optional[str] = None
    destination_taxonomy_id: Optional[int] = None
    destination_taxonomy: Optional[str] = None

    # supplier
    scraped_supplier: Optional[str] = None
    supplier_id: Optional[int] = None
    supplier: Optional[str] = None

    # charterer
    scraped_charterer: Optional[str] = None
    charterer_id: Optional[int] = None
    charterer: Optional[str] = None

    # buyer
    scraped_buyer: Optional[str] = None
    buyer_id: Optional[int] = None
    buyer: Optional[str] = None

    # port agent
    scraped_port_agent: Optional[str] = None
    port_agent_id: Optional[int] = None
    port_agent: Optional[str] = None

    # vessel status
    vessel_status_id: Optional[int] = None
    vessel_status: Optional[str] = None

    # content
    content: Optional[str] = None
    subject: Optional[str] = None

    # sender
    sender: Optional[str] = None

    # debug info
    is_private: Optional[bool] = False

ScrapedLineupsAPI

Bases: ScrapedDataAPI[ScrapedLineupsResponse, ScrapedLineup]

Represents Signal's Scraped Lineups API.

Source code in signal_ocean/scraped_lineups/scraped_lineups_api.py 
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
class ScrapedLineupsAPI(ScrapedDataAPI[ScrapedLineupsResponse, ScrapedLineup]):
    """Represents Signal's Scraped Lineups API."""

    relative_url = "scraped-lineups-api/v5.0/lineups"
    response_class = ScrapedLineupsResponse
    entity_ids_name = "lineup_ids"

    def get_lineups(
        self,
        vessel_type: int,
        message_ids: Optional[List[int]] = None,
        external_message_ids: Optional[List[str]] = None,
        received_date_from: Optional[datetime] = None,
        received_date_to: Optional[datetime] = None,
        updated_date_from: Optional[datetime] = None,
        updated_date_to: Optional[datetime] = None,
        imos: Optional[List[int]] = None,
        include_details: Optional[bool] = True,
        include_scraped_fields: Optional[bool] = True,
        include_vessel_details: Optional[bool] = True,
        include_labels: Optional[bool] = True,
        include_content: Optional[bool] = True,
        include_sender: Optional[bool] = True,
        include_debug_info: Optional[bool] = True,
    ) -> Tuple[ScrapedLineup, ...]:
        """This function collects and returns the lineups by the given filters.

        Args:
            vessel_type: Format - int32. Available values
                Tanker = 1, Dry = 3, Container = 4, Lng = 5, Lpg = 6
            message_ids: List - Comma separated list of MessageIDs
            external_message_ids: List - Comma separated list of
                ExternalMessageIDs
            received_date_from: Format - date-time (as date-time in RFC3339).
                Earliest date the lineup received.
                Cannot be combined with 'Updated' dates
            received_date_to: Format - date-time (as date-time in RFC3339).
                Latest date the lineup received.
                Cannot be combined with 'Updated' dates
            updated_date_from: Format - date-time (as date-time in RFC3339).
                Earliest date the lineup updated.
                Cannot be combined with 'Received' dates
            updated_date_to: Format - date-time (as date-time in RFC3339).
                Latest date the lineup updated.
                Cannot be combined with 'Received' dates
            imos: List - Comma separated list of IMOs
            include_details: Boolean - Whether to include
                additional lineup details in the response.
            include_scraped_fields: Boolean - Whether to include the relative
                scraped fields in the response.
            include_vessel_details: Boolean - Whether to include some vessel
                details in the response.
            include_labels: Boolean - Whether to include the relative labels in
                the response.
            include_content: Boolean - Whether to include the original message
                line (untouched) in the response.
            include_sender: Boolean - Whether to include some of the message
                sender details in the response.
            include_debug_info: Boolean - Whether to include some information
                about the distribution of the lineup in the response.

        Returns:
            An Iterable of ScrapedLineup objects, as we have defined in
            models.py Python file.
        """
        return self.get_data(
            vessel_type=vessel_type,
            message_ids=message_ids,
            external_message_ids=external_message_ids,
            received_date_from=received_date_from,
            received_date_to=received_date_to,
            updated_date_from=updated_date_from,
            updated_date_to=updated_date_to,
            imos=imos,
            include_details=include_details,
            include_scraped_fields=include_scraped_fields,
            include_vessel_details=include_vessel_details,
            include_labels=include_labels,
            include_content=include_content,
            include_sender=include_sender,
            include_debug_info=include_debug_info,
        )

    def get_lineups_by_lineup_ids(
        self,
        lineup_ids: List[int],
        include_details: Optional[bool] = True,
        include_scraped_fields: Optional[bool] = True,
        include_vessel_details: Optional[bool] = True,
        include_labels: Optional[bool] = True,
        include_content: Optional[bool] = True,
        include_sender: Optional[bool] = True,
        include_debug_info: Optional[bool] = True,
    ) -> Tuple[ScrapedLineup, ...]:
        """This function collects and returns the lineups by the given lineup ids.

        Args:
            lineup_ids: List - Comma separated list of lineup ids
            include_details: Boolean - Whether to include
                additional lineup details in the response.
            include_scraped_fields: Boolean - Whether to include the relative
                scraped fields in the response.
            include_vessel_details: Boolean - Whether to include some vessel
                details in the response.
            include_labels: Boolean - Whether to include the relative labels in
                the response.
            include_content: Boolean - Whether to include the original message
                line (untouched) in the response.
            include_sender: Boolean - Whether to include some of the message
                sender details in the response.
            include_debug_info: Boolean - Whether to include some information
                about the distribution of the lineup in the response.

        Returns:
            An Iterable of ScrapedLineup objects, as we have defined in
            models.py Python file.
        """
        return self.get_data_by_entity_ids(
            lineup_ids=lineup_ids,
            include_details=include_details,
            include_scraped_fields=include_scraped_fields,
            include_vessel_details=include_vessel_details,
            include_labels=include_labels,
            include_content=include_content,
            include_sender=include_sender,
            include_debug_info=include_debug_info,
        )

get_lineups(vessel_type, message_ids=None, external_message_ids=None, received_date_from=None, received_date_to=None, updated_date_from=None, updated_date_to=None, imos=None, include_details=True, include_scraped_fields=True, include_vessel_details=True, include_labels=True, include_content=True, include_sender=True, include_debug_info=True)

This function collects and returns the lineups by the given filters.

Parameters:

Name Type Description Default
vessel_type int

Format - int32. Available values Tanker = 1, Dry = 3, Container = 4, Lng = 5, Lpg = 6

 required
message_ids Optional[List[int]]

List - Comma separated list of MessageIDs

 None
external_message_ids Optional[List[str]]

List - Comma separated list of ExternalMessageIDs

 None
received_date_from Optional[datetime]

Format - date-time (as date-time in RFC3339). Earliest date the lineup received. Cannot be combined with 'Updated' dates

 None
received_date_to Optional[datetime]

Format - date-time (as date-time in RFC3339). Latest date the lineup received. Cannot be combined with 'Updated' dates

 None
updated_date_from Optional[datetime]

Format - date-time (as date-time in RFC3339). Earliest date the lineup updated. Cannot be combined with 'Received' dates

 None
updated_date_to Optional[datetime]

Format - date-time (as date-time in RFC3339). Latest date the lineup updated. Cannot be combined with 'Received' dates

 None
imos Optional[List[int]]

List - Comma separated list of IMOs

 None
include_details Optional[bool]

Boolean - Whether to include additional lineup details in the response.

 True
include_scraped_fields Optional[bool]

Boolean - Whether to include the relative scraped fields in the response.

 True
include_vessel_details Optional[bool]

Boolean - Whether to include some vessel details in the response.

 True
include_labels Optional[bool]

Boolean - Whether to include the relative labels in the response.

 True
include_content Optional[bool]

Boolean - Whether to include the original message line (untouched) in the response.

 True
include_sender Optional[bool]

Boolean - Whether to include some of the message sender details in the response.

 True
include_debug_info Optional[bool]

Boolean - Whether to include some information about the distribution of the lineup in the response.

 True

Returns:

Type Description
ScrapedLineup

An Iterable of ScrapedLineup objects, as we have defined in
...

models.py Python file.
Source code in signal_ocean/scraped_lineups/scraped_lineups_api.py 
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
def get_lineups(
    self,
    vessel_type: int,
    message_ids: Optional[List[int]] = None,
    external_message_ids: Optional[List[str]] = None,
    received_date_from: Optional[datetime] = None,
    received_date_to: Optional[datetime] = None,
    updated_date_from: Optional[datetime] = None,
    updated_date_to: Optional[datetime] = None,
    imos: Optional[List[int]] = None,
    include_details: Optional[bool] = True,
    include_scraped_fields: Optional[bool] = True,
    include_vessel_details: Optional[bool] = True,
    include_labels: Optional[bool] = True,
    include_content: Optional[bool] = True,
    include_sender: Optional[bool] = True,
    include_debug_info: Optional[bool] = True,
) -> Tuple[ScrapedLineup, ...]:
    """This function collects and returns the lineups by the given filters.

    Args:
        vessel_type: Format - int32. Available values
            Tanker = 1, Dry = 3, Container = 4, Lng = 5, Lpg = 6
        message_ids: List - Comma separated list of MessageIDs
        external_message_ids: List - Comma separated list of
            ExternalMessageIDs
        received_date_from: Format - date-time (as date-time in RFC3339).
            Earliest date the lineup received.
            Cannot be combined with 'Updated' dates
        received_date_to: Format - date-time (as date-time in RFC3339).
            Latest date the lineup received.
            Cannot be combined with 'Updated' dates
        updated_date_from: Format - date-time (as date-time in RFC3339).
            Earliest date the lineup updated.
            Cannot be combined with 'Received' dates
        updated_date_to: Format - date-time (as date-time in RFC3339).
            Latest date the lineup updated.
            Cannot be combined with 'Received' dates
        imos: List - Comma separated list of IMOs
        include_details: Boolean - Whether to include
            additional lineup details in the response.
        include_scraped_fields: Boolean - Whether to include the relative
            scraped fields in the response.
        include_vessel_details: Boolean - Whether to include some vessel
            details in the response.
        include_labels: Boolean - Whether to include the relative labels in
            the response.
        include_content: Boolean - Whether to include the original message
            line (untouched) in the response.
        include_sender: Boolean - Whether to include some of the message
            sender details in the response.
        include_debug_info: Boolean - Whether to include some information
            about the distribution of the lineup in the response.

    Returns:
        An Iterable of ScrapedLineup objects, as we have defined in
        models.py Python file.
    """
    return self.get_data(
        vessel_type=vessel_type,
        message_ids=message_ids,
        external_message_ids=external_message_ids,
        received_date_from=received_date_from,
        received_date_to=received_date_to,
        updated_date_from=updated_date_from,
        updated_date_to=updated_date_to,
        imos=imos,
        include_details=include_details,
        include_scraped_fields=include_scraped_fields,
        include_vessel_details=include_vessel_details,
        include_labels=include_labels,
        include_content=include_content,
        include_sender=include_sender,
        include_debug_info=include_debug_info,
    )

get_lineups_by_lineup_ids(lineup_ids, include_details=True, include_scraped_fields=True, include_vessel_details=True, include_labels=True, include_content=True, include_sender=True, include_debug_info=True)

This function collects and returns the lineups by the given lineup ids.

Parameters:

Name Type Description Default
lineup_ids List[int]

List - Comma separated list of lineup ids

 required
include_details Optional[bool]

Boolean - Whether to include additional lineup details in the response.

 True
include_scraped_fields Optional[bool]

Boolean - Whether to include the relative scraped fields in the response.

 True
include_vessel_details Optional[bool]

Boolean - Whether to include some vessel details in the response.

 True
include_labels Optional[bool]

Boolean - Whether to include the relative labels in the response.

 True
include_content Optional[bool]

Boolean - Whether to include the original message line (untouched) in the response.

 True
include_sender Optional[bool]

Boolean - Whether to include some of the message sender details in the response.

 True
include_debug_info Optional[bool]

Boolean - Whether to include some information about the distribution of the lineup in the response.

 True

Returns:

Type Description
ScrapedLineup

An Iterable of ScrapedLineup objects, as we have defined in
...

models.py Python file.
Source code in signal_ocean/scraped_lineups/scraped_lineups_api.py 
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
def get_lineups_by_lineup_ids(
    self,
    lineup_ids: List[int],
    include_details: Optional[bool] = True,
    include_scraped_fields: Optional[bool] = True,
    include_vessel_details: Optional[bool] = True,
    include_labels: Optional[bool] = True,
    include_content: Optional[bool] = True,
    include_sender: Optional[bool] = True,
    include_debug_info: Optional[bool] = True,
) -> Tuple[ScrapedLineup, ...]:
    """This function collects and returns the lineups by the given lineup ids.

    Args:
        lineup_ids: List - Comma separated list of lineup ids
        include_details: Boolean - Whether to include
            additional lineup details in the response.
        include_scraped_fields: Boolean - Whether to include the relative
            scraped fields in the response.
        include_vessel_details: Boolean - Whether to include some vessel
            details in the response.
        include_labels: Boolean - Whether to include the relative labels in
            the response.
        include_content: Boolean - Whether to include the original message
            line (untouched) in the response.
        include_sender: Boolean - Whether to include some of the message
            sender details in the response.
        include_debug_info: Boolean - Whether to include some information
            about the distribution of the lineup in the response.

    Returns:
        An Iterable of ScrapedLineup objects, as we have defined in
        models.py Python file.
    """
    return self.get_data_by_entity_ids(
        lineup_ids=lineup_ids,
        include_details=include_details,
        include_scraped_fields=include_scraped_fields,
        include_vessel_details=include_vessel_details,
        include_labels=include_labels,
        include_content=include_content,
        include_sender=include_sender,
        include_debug_info=include_debug_info,
    )