Voyages API Voyages Data Like Use Case¶

Run this example in Colab.¶

A Voyage is defined as a sequence of Load operations followed by a sequence of Discharges. Users of Signal Ocean Platform interface with the concept of a voyage in different levels of detail. For example in the Voyages tab of Vessels Data (https://app.signalocean.com/vessels) users can see all the operations of a voyage even at jetty level.
However very often arises the need of conducting an analysis of the voyages for a specific vessel class for a specific time window. This need is accommodated by the Voyages Data Dashboard (https://app.signalocean.com/reportsindex/voyagesdatalive).

The level of detail provided by the Voyages Data Dashboard has been tailored, having in mind the neccessary information needed to carry out such an analysis without being overwhelmed by the full data provided by Signal Ocean Platform regarding the voyages of the vessels.

While both get_voyages and get_voyages_flat functions of the Signal SDK return the full low level data available, in this example we are going to construct a dataframe that resembles the form of Voyages Data Dashboard

Setup¶

Install the Signal Ocean SDK:

pip install signal-ocean

Set your subscription key acquired here: https://apis.signalocean.com/profile

In [ ]:

pip install signal-ocean

pip install signal-ocean

In [ ]:

signal_ocean_api_key = '' #replace with your subscription key

signal_ocean_api_key = '' #replace with your subscription key

In [ ]:

from signal_ocean import Connection
from signal_ocean.voyages import VoyagesAPI
from signal_ocean.voyages import VesselClass, VesselClassFilter
import pandas as pd
import numpy as np
from datetime import date, timedelta, datetime, timezone
from dateutil.relativedelta import relativedelta

from signal_ocean import Connection from signal_ocean.voyages import VoyagesAPI from signal_ocean.voyages import VesselClass, VesselClassFilter import pandas as pd import numpy as np from datetime import date, timedelta, datetime, timezone from dateutil.relativedelta import relativedelta

In [ ]:

pd.set_option('display.max_columns', None)

pd.set_option('display.max_columns', None)

In [ ]:

connection = Connection(signal_ocean_api_key)
api = VoyagesAPI(connection)

connection = Connection(signal_ocean_api_key) api = VoyagesAPI(connection)

Get voyages¶

For this tutorial we will retrieve the voyages of VLCC vessels that have started during the last semester of 2021.

In [6]:

#get vessel class id for vlcc
vc = api.get_vessel_classes(VesselClassFilter('vlcc'))[0]
vlcc_id = vc.vessel_class_id
vlcc_id

#get vessel class id for vlcc vc = api.get_vessel_classes(VesselClassFilter('vlcc'))[0] vlcc_id = vc.vessel_class_id vlcc_id

Out[6]:

84

In [7]:

start_date_to = date(2021,12,31)
start_date_from = start_date_to - relativedelta(months=6)

start_date_to = date(2021,12,31) start_date_from = start_date_to - relativedelta(months=6)

In [8]:

voyages = api.get_voyages_by_advanced_search(
    vessel_class_id=vlcc_id, 
    start_date_from=start_date_from,
    start_date_to = start_date_to
)

voyages = api.get_voyages_by_advanced_search( vessel_class_id=vlcc_id, start_date_from=start_date_from, start_date_to = start_date_to )

In [9]:

voyages = pd.DataFrame(v.__dict__ for v in voyages)
events = pd.DataFrame(e.__dict__ for voyage_events in voyages['events'].dropna() for e in voyage_events)

voyages = pd.DataFrame(v.__dict__ for v in voyages) events = pd.DataFrame(e.__dict__ for voyage_events in voyages['events'].dropna() for e in voyage_events)

In [10]:

# we filter out voyages that have no actual load and discharge port calls
# (current voyages for ballast unfixed vessels)
voyages.end_date = pd.to_datetime(voyages.end_date, errors = 'coerce', utc = True)
voyages.dropna(subset = ['end_date'], inplace = True)

# we filter out voyages that have no actual load and discharge port calls # (current voyages for ballast unfixed vessels) voyages.end_date = pd.to_datetime(voyages.end_date, errors = 'coerce', utc = True) voyages.dropna(subset = ['end_date'], inplace = True)

In [11]:

def get_open_load_discharge_events(voyage_events):
    open_event = next((e.__dict__ for e in voyage_events or [] if e.purpose=='Start'), None)
    load_event = next((e.__dict__ for e in voyage_events or [] if e.purpose=='Load'), None)
    discharge_event = next((e.__dict__ for e in reversed(voyage_events) or [] if e.purpose=='Discharge'), None)
    return pd.Series((open_event,load_event, discharge_event))
    
voyages[['open_event','load_event','discharge_event']] = voyages['events'].apply(get_open_load_discharge_events)

def get_open_load_discharge_events(voyage_events): open_event = next((e.__dict__ for e in voyage_events or [] if e.purpose=='Start'), None) load_event = next((e.__dict__ for e in voyage_events or [] if e.purpose=='Load'), None) discharge_event = next((e.__dict__ for e in reversed(voyage_events) or [] if e.purpose=='Discharge'), None) return pd.Series((open_event,load_event, discharge_event)) voyages[['open_event','load_event','discharge_event']] = voyages['events'].apply(get_open_load_discharge_events)

In [12]:

mapping_dict = {'port_name':['starting_port','first_load_port','last_discharge_port'],
                'area_name_level0':['starting_area','first_load_area','last_discharge_area'], 
                'country':['starting_country','first_load_country','last_discharge_country'],
                'arrival_date':['open_port_arrival_date','first_load_port_arrival_date','last_discharge_port_arrival_date'],
                'sailing_date':['open_port_sailing_date','first_load_port_sailing_date','last_discharge_port_sailing_date'], 
                }

events = {0:'open_event',1:'load_event',2:'discharge_event'}

mapping_dict = {'port_name':['starting_port','first_load_port','last_discharge_port'], 'area_name_level0':['starting_area','first_load_area','last_discharge_area'], 'country':['starting_country','first_load_country','last_discharge_country'], 'arrival_date':['open_port_arrival_date','first_load_port_arrival_date','last_discharge_port_arrival_date'], 'sailing_date':['open_port_sailing_date','first_load_port_sailing_date','last_discharge_port_sailing_date'], } events = {0:'open_event',1:'load_event',2:'discharge_event'}

In [13]:

for feature,targets in mapping_dict.items():
    for num,target in enumerate(targets):
        voyages[target] = voyages[events[num]].apply(lambda e: e[feature] if isinstance(e,dict) else None)

for feature,targets in mapping_dict.items(): for num,target in enumerate(targets): voyages[target] = voyages[events[num]].apply(lambda e: e[feature] if isinstance(e,dict) else None)

In [14]:

def get_start_time_of_operation(event):
    if (event['event_type'] == 'PortCall') and (event['event_horizon'] != 'Future'):
        next_event_detail = next((ed.__dict__ for ed in event['event_details'] or []), None)
        return next_event_detail['start_time_of_operation']

def get_start_time_of_operation(event): if (event['event_type'] == 'PortCall') and (event['event_horizon'] != 'Future'): next_event_detail = next((ed.__dict__ for ed in event['event_details'] or []), None) return next_event_detail['start_time_of_operation']

In [15]:

voyages.loc[voyages.load_event.notna(),'first_load_port_start_time_of_operation'] = (
   voyages.loc[voyages.load_event.notna()].load_event.apply(get_start_time_of_operation)
)
voyages.loc[voyages.load_event.notna(),'last_discharge_port_start_time_of_operation'] = (
   voyages.loc[voyages.discharge_event.notna()].discharge_event.apply(get_start_time_of_operation)
)

voyages.first_load_port_start_time_of_operation = pd.to_datetime(voyages.first_load_port_start_time_of_operation)
voyages.last_discharge_port_start_time_of_operation = pd.to_datetime(voyages.last_discharge_port_start_time_of_operation)

voyages.loc[voyages.load_event.notna(),'first_load_port_start_time_of_operation'] = ( voyages.loc[voyages.load_event.notna()].load_event.apply(get_start_time_of_operation) ) voyages.loc[voyages.load_event.notna(),'last_discharge_port_start_time_of_operation'] = ( voyages.loc[voyages.discharge_event.notna()].discharge_event.apply(get_start_time_of_operation) ) voyages.first_load_port_start_time_of_operation = pd.to_datetime(voyages.first_load_port_start_time_of_operation) voyages.last_discharge_port_start_time_of_operation = pd.to_datetime(voyages.last_discharge_port_start_time_of_operation)

In [16]:

def get_sts_load_ind(load_event):
    return next((True for d in load_event["event_details"] or [] if d.event_detail_type =='StS'), False)

def get_sts_discharge_ind(discharge_event):
    return next((True for d in discharge_event["event_details"] or [] if d.event_detail_type =='StS'), False)


voyages.loc[voyages.discharge_event.notna(),'sts_discharge_ind'] = \
voyages.loc[voyages.discharge_event.notna(),'discharge_event'].apply(get_sts_discharge_ind)
voyages.loc[voyages.load_event.notna(),'sts_load_ind'] = \
voyages.loc[voyages.load_event.notna(),'load_event'].apply(get_sts_load_ind)

def get_sts_load_ind(load_event): return next((True for d in load_event["event_details"] or [] if d.event_detail_type =='StS'), False) def get_sts_discharge_ind(discharge_event): return next((True for d in discharge_event["event_details"] or [] if d.event_detail_type =='StS'), False) voyages.loc[voyages.discharge_event.notna(),'sts_discharge_ind'] = \ voyages.loc[voyages.discharge_event.notna(),'discharge_event'].apply(get_sts_discharge_ind) voyages.loc[voyages.load_event.notna(),'sts_load_ind'] = \ voyages.loc[voyages.load_event.notna(),'load_event'].apply(get_sts_load_ind)

In [17]:

def get_repairs_ind(events):
    for ev in events:
        if ev.purpose == 'Dry dock':
            return True
    return False

def get_repairs_ind(events): for ev in events: if ev.purpose == 'Dry dock': return True return False

In [18]:

voyages['repairs_ind'] = voyages.events.apply(get_repairs_ind)

voyages['repairs_ind'] = voyages.events.apply(get_repairs_ind)

In [19]:

def get_storage_ind(events):
    for ev in events:
        if ev.purpose == 'StorageVessel':
            return True
    return False

def get_storage_ind(events): for ev in events: if ev.purpose == 'StorageVessel': return True return False

In [20]:

voyages['storage_ind'] = voyages.events.apply(get_storage_ind)

voyages['storage_ind'] = voyages.events.apply(get_storage_ind)

In [21]:

voyages['local_trade_ind'] = voyages.apply(
    lambda row: row['first_load_country'] == row['last_discharge_country'],
    axis = 1
)

voyages['local_trade_ind'] = voyages.apply( lambda row: row['first_load_country'] == row['last_discharge_country'], axis = 1 )

In [22]:

vessel_status_dict = {
    1:"Voyage", 2:"Breaking", 3:"Domestic Trade", 4:"FPSO", 5:"FPSO Conversion", 
    6:"Inactive", 7:"Storage Vessel", 9:"Conversion"
}
voyages['vessel_status'] = voyages.vessel_status_id.replace(vessel_status_dict)

vessel_status_dict = { 1:"Voyage", 2:"Breaking", 3:"Domestic Trade", 4:"FPSO", 5:"FPSO Conversion", 6:"Inactive", 7:"Storage Vessel", 9:"Conversion" } voyages['vessel_status'] = voyages.vessel_status_id.replace(vessel_status_dict)

In [23]:

commercial_status_dict = {
    0:"OnSubs", 1:"FullyFixed", 2:"Failed", 3:"Cancelled", 4:"Available", 
    -1:"Unknown", -2:"NotSet"
}
voyages['commercial_status'] = voyages.fixture_status_id.replace(commercial_status_dict)

commercial_status_dict = { 0:"OnSubs", 1:"FullyFixed", 2:"Failed", 3:"Cancelled", 4:"Available", -1:"Unknown", -2:"NotSet" } voyages['commercial_status'] = voyages.fixture_status_id.replace(commercial_status_dict)

In [24]:

wanted_columns = ['vessel_name',
                  'imo',
                  'vessel_class',
                  'commercial_operator',
                  'voyage_number',
                  'start_date',
                  'end_date',
                  'starting_port',
                  'first_load_port',
                  'last_discharge_port',
                  'first_load_port_arrival_date',
                  'first_load_port_start_time_of_operation',
                  'first_load_port_sailing_date',
                  'last_discharge_port_arrival_date',
                  'last_discharge_port_start_time_of_operation',
                  'last_discharge_port_sailing_date',
                  'charterer',
                  'rate',
                  'rate_type',
                  'laycan_from',
                  'laycan_to',
                  'quantity',
                  'cargo_group',
                  'cargo_type',
                  'cargo_type_source',
                  'fixture_is_coa',
                  'fixture_is_hold',
                  'fixture_date',
                  'trade',
                  'vessel_status',
                  'commercial_status',
                  'starting_country',
                  'starting_area',
                  'first_load_country',
                  'first_load_area',
                  'last_discharge_country',
                  'last_discharge_area',
                  'sts_load_ind',
                  'sts_discharge_ind',
                  'storage_ind',
                  'repairs_ind',
                  'is_implied_by_ais',
                  'local_trade_ind',
                  'has_manual_entries',
                  'ballast_distance',
                  'laden_distance'
                 ]

voyages = voyages[wanted_columns]

wanted_columns = ['vessel_name', 'imo', 'vessel_class', 'commercial_operator', 'voyage_number', 'start_date', 'end_date', 'starting_port', 'first_load_port', 'last_discharge_port', 'first_load_port_arrival_date', 'first_load_port_start_time_of_operation', 'first_load_port_sailing_date', 'last_discharge_port_arrival_date', 'last_discharge_port_start_time_of_operation', 'last_discharge_port_sailing_date', 'charterer', 'rate', 'rate_type', 'laycan_from', 'laycan_to', 'quantity', 'cargo_group', 'cargo_type', 'cargo_type_source', 'fixture_is_coa', 'fixture_is_hold', 'fixture_date', 'trade', 'vessel_status', 'commercial_status', 'starting_country', 'starting_area', 'first_load_country', 'first_load_area', 'last_discharge_country', 'last_discharge_area', 'sts_load_ind', 'sts_discharge_ind', 'storage_ind', 'repairs_ind', 'is_implied_by_ais', 'local_trade_ind', 'has_manual_entries', 'ballast_distance', 'laden_distance' ] voyages = voyages[wanted_columns]

In [25]:

import re

def snake_to_camel(word):
    return ''.join(x.capitalize() or '_' for x in word.split('_'))

import re def snake_to_camel(word): return ''.join(x.capitalize() or '_' for x in word.split('_'))

In [26]:

voyages.columns = [*map(snake_to_camel, voyages.columns)]
voyages

voyages.columns = [*map(snake_to_camel, voyages.columns)] voyages

Out[26]:

	VesselName	Imo	VesselClass	CommercialOperator	VoyageNumber	StartDate	EndDate	StartingPort	FirstLoadPort	LastDischargePort	...	LastDischargeArea	StsLoadInd	StsDischargeInd	StorageInd	RepairsInd	IsImpliedByAis	LocalTradeInd	HasManualEntries	BallastDistance	LadenDistance
0	Artemis III	9102241	VLCC	Bahri	132	2021-09-27 13:22:23.500000+00:00	2023-04-20 19:56:09+00:00	Dongjiangkou	Singapore	Singapore	...	Singapore / Malaysia	False	False	False	False	None	True	None	1650.51	14979.22
1	FT Island	9166675	VLCC	None	43	2021-08-25 01:41:55+00:00	2023-04-18 15:58:33+00:00	Huangpu	Singapore	Tanjung Pelepas	...	Singapore / Malaysia	False	True	False	True	None	False	None	9911.22	920.89
2	Lisa	9174397	VLCC	None	47	2021-12-20 20:04:34+00:00	2022-12-14 03:56:21+00:00	Qingdao	Malongo	Tianjin	...	North China	False	False	False	False	None	False	None	18788.49	10730.73
3	Destiny	9177155	VLCC	NITC	26	2021-11-19 15:55:24+00:00	2023-05-04 13:45:06+00:00	Tranmere	Pazflor Oil Field	Cotonou	...	Africa Atlantic Coast	False	False	False	False	None	False	None	4502.32	1068.07
4	Roma	9182291	VLCC	New Shipping	67	2021-07-28 15:59:44+00:00	2021-09-25 01:50:26+00:00	Tanjung Pelepas	Fujairah	Tanjung Pelepas	...	Singapore / Malaysia	False	True	False	False	None	False	None	3322.17	3204.19
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
1881	Tateshina	9910117	VLCC	None	1	2021-10-27 01:04:54+00:00	2022-01-09 07:55:52+00:00	Qushan Island	Ruwais	Rotterdam	...	Continent	False	False	False	True	None	False	None	6216.48	6651.49
1882	Towa Maru	9910181	VLCC	None	1	2021-11-14 23:57:53+00:00	2022-03-15 19:57:27+00:00	Kure, Hiroshima	Fujairah	Yokkaichi	...	Japan Island	False	False	False	True	None	False	None	7393.28	7276.41
1883	Julius Caesar	9912244	VLCC	None	1	2021-12-17 06:20:23+00:00	2022-03-16 15:58:35+00:00	Ulsan	Vizhinjam	Lome	...	Africa Atlantic Coast	True	True	False	True	None	False	None	5536.30	7239.92
1884	Grand Bonanza	9915569	VLCC	Koch	1	2021-10-20 05:16:43+00:00	2022-02-24 05:29:45+00:00	Okpo/Geoje	Ruwais	Lome	...	Africa Atlantic Coast	False	True	False	True	None	False	None	7651.64	7965.27
1885	Mpf3	9919436	VLCC	None	1	2021-12-12 12:46:01+00:00	2023-08-06 16:18:08+00:00	Shanghai	Galang Island Oil Field	Rotterdam	...	Continent	False	False	False	True	None	False	None	2270.15	6359.17

1881 rows × 46 columns

In [27]:

datetime_columns = voyages.select_dtypes(include=['datetime64[ns, UTC]']).columns

voyages.loc[:,datetime_columns] = (
    voyages
    .select_dtypes(
        include=['datetime64[ns, UTC]']
    ).apply(lambda column: column.dt.tz_localize(None),
        axis = 0
    )
)

datetime_columns = voyages.select_dtypes(include=['datetime64[ns, UTC]']).columns voyages.loc[:,datetime_columns] = ( voyages .select_dtypes( include=['datetime64[ns, UTC]'] ).apply(lambda column: column.dt.tz_localize(None), axis = 0 ) )

In [28]:

voyages.to_excel('voyages_data.xlsx', index = False)

voyages.to_excel('voyages_data.xlsx', index = False)