AskOmics

AskOmics is a visual SPARQL query interface supporting both intuitive data integration and querying while shielding the user from most of the technical difficulties underlying RDF and SPARQL

Deployment

User

Manual installation

Dependencies
Virtuoso

AskOmics work with an RDF triplestore.

Compile virtuoso or install it with docker

Python3/venv/npm

AskOmics is build in python3 and javascript. Install the following packages

# ubuntu
apt install -y python3 python3-venv npm
# Fedora
dnf install -y python3 python3-venv npm
Installation with scripts

Install AskOmics with install.sh and run it with run_all.sh. AskOmics will be available at localhost:5000.

Installation with docker-compose

Clone the askomics-docker-compose repository

# clone
git clone https://github.com/askomics/flaskomics-docker-compose.git
# cd
cd flaskomics-docker-compose/standalone
# Or, if you want to use federated queries
cd flaskomics-docker-compose/federated

Update config (see README)

Run

sudo docker-compose up -d

AskOmics will be available at localhost.

Installation with a single docker

Docker image askomics/flaskomics-with-dependencies contain AskOmics with all his dependencies (Redis, virtuoso, celery …).

# Pull image
docker pull askomics/flaskomics-with-dependencies
# run image
docker run -d askomics/flaskomics-with-dependencies

If you need a persistent volume, run

docker run -d -v ./flaskomics-data:/tmp/flaskomics askomics/flaskomics-with-dependencies

The image create a default user at the first run. You can update this user by setting the following environment variables:

|ENV|User field|default value| |—-|—-|—-| |USER_FIRST_NAME|First name|Ad| |USER_LAST_NAME|Last name|Min| |USERNAME|Username|admin| |USER_EMAIL|Email|admin@example.org| |USER_PASSWORD|Password (clear)|admin| |USER_APIKEY|User api key|admin| |GALAXY_URL|Galaxy url (optional)| | |GALAXY_API_KEY|Galaxy api key (optional)| |

For example:

docker run -d -v ./flaskomics-data:/tmp/flaskomics -e USER_FIRST_NAME="John" -e USER_LAST_NAME="Wick" -e USERNAME="jwick" askomics/flaskomics-with-dependencies

Developer

AskOmics

Fork the AskOmics repository

then, clone your fork

git clone https://github.com/USERNAME/flaskomics.git # replace USERNAME with your github username

Install AskOmics

Run it with dev mod

./run_all.sh -d dev

AskOmics will be available at localhost:5000

Dependencies

Clone the askomics-docker-compose repository and go into the dev directory.

# clone
git clone https://github.com/askomics/flaskomics-docker-compose.git
# cd
cd flaskomics-docker-compose/dev

The dev directory contain a docker-compose.yml file who can lauch all dependencies needed by AskOmics (Virtuoso, Redis, Corese, Galaxy).

# clone
docker-compose up -d

Use the ci directory to lauch dockers for testing AskOmics

cd ../ci
docker-compose up -d

Usage

TODO: Write the docs ;-)

Use with external endpoint

AskOmics can be use to explore external endpoint such as NeXtProt.

Build external endpoint abstraction

First, build AskOmics abstraction of the external enpoint with Abstractor.

Install Abstractor in a python virtual env

# Create and source venv
python3 -m venv venv
source venv/bin/activate
# Install Abstractor
pip install abstractor

Generate AskOmics abstraction. The tool explore the entire endpoint, so it can be long.

# Need help ?
abstractor -h
# Generate abstraction
abstractor -e <enpoint_url> -p <main_prefix> -o abstraction.ttl

Deploy AskOmics

Deploy AskOmics with a special configuration to use it with an external endpoint. The following ini entry have to be updated:

  • askomics
    • disable_integration: true
  • triplestore
    • external_endpoint: <endpoint_url>
    • prefix = <main_prefix>
    • namespace = <main_prefix>

Integrate external endpoint

Create an account on the AskOmics instance. The first account is an admin account who can integrate datasets. Load the asbtraction.ttl and integrate it has a public dataset.

Other users can’t integrate datasets, but they can explore the external endpoint.

Contribute to AskOmics

Issues

If you have an idea for a feature to add or an approach for a bugfix, it is best to communicate with developers early. The most common venues for this are GitHub issues.

Pull requests

All changes to AskOmics should be made through pull requests to this repository.

For the askomics repository to your account. To keep your copy up to date, you need to frequently sync your fork:

git remote add upstream https://github.com/askomics/flaskomics
git fetch upstream
git checkout master
git merge upstream/master

Then, create a new branch for your new feature

git checkout -b my_new_feature

Commit and push your modification to your fork. If your changes modify code, please ensure that is conform to AskOmics style

Write tests for your changes, and make sure that they passes.

Open a pull request against the master branch of askomics. The message of your pull request should describe your modifications (why and how).

The pull request should pass all the continuous integration tests which are automatically run by Github using Travis CI. The coverage must be at least remain the same (but it’s better if it increases)

Tests

Use test.sh script to lint and test the code. Don’t PR if linting or testing don’t pass.

Coding style guidelines

General

Ensure all user-enterable strings are unicode capable. Use only English language for everything (code, documentation, logs, comments, …)

Python

We follow PEP-8, with particular emphasis on the parts about knowing when to be inconsistent, and readability being the ultimate goal.

  • Whitespace around operators and inside parentheses
  • 4 spaces per indent, spaces, not tabs
  • Include docstrings on your modules, class and methods
  • Avoid from module import *. It can cause name collisions that are tedious to track down.
  • Class should be in CamelCase, methods and variables in lowercase_with_underscore

Contribute to docs

all the documentation (including what you are reading) can be found here. Files are on the AskOmics repository.

To preview the docs, run

cd flaskomics
# source the askomics virtual env
source venv/bin/activate
cd docs
make html

html files are in build directory.

askomics package

Subpackages

askomics.api package

Submodules
askomics.api.admin module

Admin routes

askomics.api.admin.get_users()

Get all users

Returns:users: list of all users info error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.admin.set_admin()

change admin status of a user

Returns:error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.admin.set_blocked()

Change blocked status of a user

Returns:error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.admin.set_quota()

Change quota of a user

Returns:users: updated users error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.auth module

Authentication routes

askomics.api.auth.admin_required(f)

Login required function

askomics.api.auth.login()

Log a user

Returns:Information about the logged user
Return type:json
askomics.api.auth.login_api_key(key)

Log user with his API key

Parameters:key (string) – User API key
Returns:Information about the logged user
Return type:json
askomics.api.auth.login_required(f)

Login required function

askomics.api.auth.logout()

Logout the current user

Returns:no username and logged false
Return type:json
askomics.api.auth.signup()

Register a new user

Returns:Info about the user
Return type:json
askomics.api.auth.update_apikey()

Update the user apikey

Returns:The user with his new apikey
Return type:json
askomics.api.auth.update_galaxy()

Update the user apikey

Returns:The user with his new apikey
Return type:json
askomics.api.auth.update_password()

Update the user passord

Returns:The user
Return type:json
askomics.api.auth.update_profile()

Update user profile (names and email)

Returns:The updated user
Return type:json
askomics.api.catch_url module

Catch_all route

askomics.api.catch_url.catch_all(path)

Return all routes to home

Parameters:path (str) – Original path
Returns:Redirect to route /
Return type:redirect
askomics.api.datasets module

Api routes

askomics.api.datasets.delete_datasets()

Delete some datasets (db and triplestore) with a celery task

Returns:error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.datasets.get_datasets()

Get datasets information

Returns:datasets: list of all datasets of current user error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.datasets.toogle_public()

Toggle public status of a dataset

Returns:error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.file module

Api routes

askomics.api.file.delete_files()

Delete files

Returns:files: list of all files of current user error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.file.get_files()

Get files info of the logged user

Returns:files: list of all files of current user error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.file.get_preview()

Get files preview

Returns:previewFiles: preview of selected files error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.file.integrate()

Integrate a file

Returns:task_id: celery task id error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.file.serve_file(path, user_id, username)

Serve a static ttl file of a user

Parameters:
  • path (string) – The file path to serve
  • user_id (int) – user id
  • username (string) – username
Returns:

the file
Return type:

file
askomics.api.file.upload_chunk()

Upload a file chunk

Returns:path: name of the local file. To append the next chunk into it error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.file.upload_url()

Upload a distant file with an URL

Returns:error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.galaxy module

Api routes

askomics.api.galaxy.get_dataset_content()

Download a galaxy datasets into AskOmics

Returns:dataset_content: content of the requested Galaxy dataset error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.galaxy.get_datasets()

Get galaxy datasets and histories of a user

Returns:error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.galaxy.get_queries()

Get galaxy queries (json datasets)

Returns:error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.galaxy.upload_datasets()

Download a galaxy datasets into AskOmics

Returns:error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.query module
askomics.api.query.get_abstraction()

Get abstraction

Returns:abstraction: abstraction error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.query.get_preview()

Get a preview of query

Returns:resultsPreview: Preview of the query results headerPreview: Header of the results table error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.query.query()

Get start points

Returns:startpoint: list of start points error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.query.save_result()

Save a query in filesystem and db, using a celery task

Returns:task_id: celery task id error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.results module
askomics.api.results.delete_result()

Summary

Returns:files: list of all files of current user error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.results.download_result()

Download result file

askomics.api.results.get_graph_state()

Summary

Returns:preview: list of result preview header: result header error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.results.get_preview()

Summary

Returns:preview: list of result preview header: result header error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.results.get_results()

Get …

Returns:files: list of all files of current user error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.results.get_sparql_query()

Get sparql query of result for the query editor

Returns:query: the sparql query error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.results.publish_query()

Publish a query template from a result

Returns:error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.results.send2galaxy()

Send a result file into Galaxy

Returns:error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.results.set_description()

Update a result description

Returns:files: all files error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.results.set_public()

Change public status of a file, and return all files

Returns:files: list of all files of current user error: True if error, else False errorMessage: the error message of error, else an empty string
Return type:json
askomics.api.sparql module
askomics.api.sparql.prefix()

Get the default sparql query

Returns:default query
Return type:json
askomics.api.sparql.query()

Perform a sparql query

Returns:query results
Return type:json
askomics.api.sparql.save_query()

Perform a sparql query

Returns:query results
Return type:json
askomics.api.start module
askomics.api.start.hello()

Dummy routes

Returns:error: True if error, else False errorMessage: the error message of error, else an empty string message: a welcome message
Return type:json
askomics.api.start.start()

Starting route

Returns:Information about a eventualy logged user, and the AskOmics version and a footer message
Return type:json
askomics.api.view module

Render route

askomics.api.view.home(path)

Render the html of AskOmics

Returns:Html code of AskOmics
Return type:html
Module contents

askomics.libaskomics package

Submodules
askomics.libaskomics.BedFile module
class askomics.libaskomics.BedFile.BedFile(app, session, file_info, host_url=None, external_endpoint=None, custom_uri=None)

Bases: askomics.libaskomics.File.File

Bed File

public

Public or private dataset

Type:bool
generate_rdf_content()

Generate RDF content of the BED file

Yields:Graph – RDF content
get_preview()

Get file preview

Returns:bed file preview
Return type:dict
integrate(entity_name, public=True)

Integrate BeD file

Parameters:
  • entities (List) – Entities to integrate
  • public (bool, optional) – Insert in public dataset
set_preview()

Set entity name preview

set_rdf_abstraction_domain_knowledge()

Set the abstraction and domain knowledge

askomics.libaskomics.CsvFile module
class askomics.libaskomics.CsvFile.CsvFile(app, session, file_info, host_url=None, external_endpoint=None, custom_uri=None)

Bases: askomics.libaskomics.File.File

CSV file

category_values

Category values

Type:dict
columns_type

Columns type

Type:list
header

Header

Type:list
preview

Previex

Type:list
public

Public

Type:bool
check_columns_types()

Check all columns type after detection and correct them

dialect

Like @property on a member function, but also cache the calculation in self.__dict__[function name]. The function is called only once since the cache stored as an instance attribute override the property residing in the class attributes. Following accesses cost no more than standard Python attribute access. If the instance attribute is deleted the next access will re-evaluate the function. Source: https://blog.ionelmc.ro/2014/11/04/an-interesting-python-descriptor-quirk/

class Shape(object):

@cached_property def area(self):

# compute value return value
func

Description

Type:TYPE
force_columns_type(forced_columns_type)

Set the columns type without detecting them

Parameters:forced_columns_type (list) – columns type
generate_rdf_content()

Generator of the rdf content

Yields:Graph – Rdf content
get_preview()

Get a preview of the file

Returns:File preview
Return type:dict
guess_column_type(values, header_index)

Guess the columns type

Parameters:
  • values (list) – columns preview
  • header_index (int) – Header index
Returns:

The guessed type
Return type:

string
integrate(forced_columns_type, public=False)

Integrate the file

Parameters:
  • forced_columns_type (list) – columns type
  • public (bool, optional) – True if dataset will be public
is_category(values)

Check if a list af values are categories

Parameters:values (list) – List of values
Returns:True if values are categories
Return type:bool
static is_decimal(value)

Guess if a variable if a number

Parameters:value – The var to test
Returns:True if it’s decimal
Return type:boolean
set_columns_type()

Set the columns type by guessing them

set_preview()

Set previex, header and columns type by sniffing the file

set_preview_and_header()

Set the preview and header by looking in the fists lines of the file

set_rdf_abstraction()

Set the abstraction

set_rdf_abstraction_domain_knowledge()

Set intersection of abstraction and domain knowledge

set_rdf_domain_knowledge()

Set the domain knowledge

transposed_preview

Transpose the preview

Returns:Transposed preview
Return type:list
askomics.libaskomics.Database module

Contain the Database class

class askomics.libaskomics.Database.Database(app, session)

Bases: askomics.libaskomics.Params.Params

Manage Database connection

database_path

Path to the database file

Type:str
create_datasets_table()

Create the datasets table

create_endpoints_table()

Create the endpoints table

create_files_table()

Create the files table

create_galaxy_table()

Create the galaxy table

create_integration_table()

Create the integration table

create_results_table()

Create the results table

create_user_table()

Create the user table

execute_sql_query(query, variables=[], get_id=False)

Execute an sql query to the database

Parameters:
  • query (str) – The sql query
  • variables (List, optional) – Sql variables
  • get_id (bool, optional) – Return the last row id
Returns:

Result of the query, or last row id
Return type:

List
init_database()

Create all tables

update_datasets_table()

Add cols on the datasets table

update_results_table()

Add the size and sparql_query cols on the results table

update_users_table()

Add the quota col on the users table

Update the users table for the instance who don’t have this column

askomics.libaskomics.Dataset module
class askomics.libaskomics.Dataset.Dataset(app, session, dataset_info={})

Bases: askomics.libaskomics.Params.Params

celery_id

celery id

Type:string
file_id

database file id

Type:int
graph_name

graph name

Type:string
id

database dataset id

Type:int
name

dataset name

Type:string
public

Public

Type:bool
delete_from_db()

Delete a dataset from the database

save_in_db()

Save the dataset into the database

set_info_from_db()

Set the info in from the database

toggle_public(new_status)

Change public status of a dataset (triplestore and db)

Parameters:new_status (bool) – True if public
update_celery(celery_id)

Update celery id of dataset in database

Parameters:celery_id (string) – DescriThe celery idption
update_in_db(status, update_celery=False, error=False, error_message=None, ntriples=0, traceback=None)

Update the dataset when integration is done

Parameters:
  • error (bool, optional) – True if error during integration
  • error_message (None, optional) – Error string if error is True
  • ntriples (int, optional) – Number of triples integrated
askomics.libaskomics.DatasetsHandler module
class askomics.libaskomics.DatasetsHandler.DatasetsHandler(app, session, datasets_info=[])

Bases: askomics.libaskomics.Params.Params

Summary

datasets

Description

Type:list
datasets_info

Description

Type:TYPE
delete_datasets()

delete the datasets from the database and the triplestore

delete_datasets_in_db()

Delete datasets of the database

get_datasets()

Get info about the datasets

Returns:Datasets informations
Return type:list of dict
handle_datasets()

Handle datasets

update_status_in_db(status)

Update the status of a datasets in the database

Parameters:status (string) – The new status (started, success or deleting)
Returns:Remaining datasets
Return type:list
askomics.libaskomics.File module
class askomics.libaskomics.File.File(app, session, file_info, host_url=None, external_endpoint=None, custom_uri=None)

Bases: askomics.libaskomics.Params.Params

Summary

askomics_namespace

AskOmics namespace askomics:

Type:Namespace
askomics_prefix

AskOmics prefix :

Type:Namespace
dc

dc namespace

Type:Namespace
default_graph

Default rdf graph

Type:string
faldo

faldo namespace

Type:Namespace
faldo_entity

True if entity is a faldo entity

Type:bool
file_graph

File graph containing the file

Type:string
host_url

AskOmics url

Type:string
id

database file id

Type:int
max_chunk_size

Max number of triple to insert in one Load or insert

Type:int
method

Load or insert

Type:int
name

Name of the file

Type:string
now

timestamp of the current time

Type:datetime
ntriples

Number of triples

Type:int
path

Path of the file

Type:string
prov

prov namespace

Type:Namespace
public

True if the file is public

Type:bool
size

file size

Type:int
timestamp

Description

Type:TYPE
ttl_dir

path to the ttl directory

Type:string
type

file type

Type:string
user_graph

User graph

Type:string
convert_type(value)

Convert a value to a int or float or text

Parameters:value (string) – The value to convert
Returns:the converted value
Return type:string/float/int
format_uri(string, remove_space=False)

remove space and quote

get_faldo_strand(raw_strand)

Get faldo strand

Parameters:raw_strand (string) – raw value of strand
Returns:Faldo “Foward”, “Reverse” or “Both” uri
Return type:rdf term
get_metadata()

Get a rdflib graph of the metadata

Returns:graph containing metadata of the file
Return type:Graph
get_rdf_type(value)

get xsd type of a value

Parameters:value – The value to get type
Returns:rdflib.XSD.string or rdflib.XSD.decimal
Return type:TYPE
integrate()

Integrate the file into the triplestore

load_graph(rdf_graph, tmp_file_name)

Load a rdflib graph into the triplestore

Write rdf to a tmp file, and send the url to this file to the triplestore with a LOAD request

Parameters:
  • rdf_graph (Graph) – rdf graph to load
  • tmp_file_name (string) – Path to a tmp file
rdfize(string)

Rdfize a string

Return the literal is string is an url, else, prefix it with askomics prefix

Parameters:string (string) – Term to rdfize
Returns:Rdfized term
Return type:rdflib.???
rollback()

Drop the dataset from the triplestore in case of error

set_triples_number()

Set graph triples number by requesting the triplestore

askomics.libaskomics.FilesHandler module
class askomics.libaskomics.FilesHandler.FilesHandler(app, session, host_url=None, external_endpoint=None, custom_uri=None)

Bases: askomics.libaskomics.FilesUtils.FilesUtils

Handle files

files

list of File

Type:list
host_url

AskOmics url, for the triplestore

Type:string
upload_path

Upload path

Type:string
delete_file_from_db(file_id)

remove a file for the database

Parameters:file_id (int) – the file id to remove
delete_file_from_fs(file_path)

Delete a file from filesystem

Parameters:file_path (string) – Path to the file
delete_files(files_id)

Delete files from database and filesystem

Parameters:files_id (list) – list of file id
Returns:list of files info
Return type:list
download_url(url)

Download a file from an URL and insert info in database

Parameters:url (string) – The file url
get_file_name()

Get a random file name

Returns:file name
Return type:string
get_file_path(file_id)

Get the file path with id

Parameters:file_id (int) – the file id
Returns:file path
Return type:string
get_files_infos(files_id=None, return_path=False)

Get files info

Parameters:
  • files_id (None, optional) – list of files id
  • return_path (bool, optional) – return the path if True
Returns:

list of files info
Return type:

list
get_type(file_ext)

Get files type, based on extension

TODO: sniff file to get type

Parameters:file_ext (string) – file extension
Returns:file type
Return type:string
handle_files(files_id)

Handle file

Parameters:files_id (list) – id of files to handle
persist_chunk(chunk_info)

Persist a file by chunk. Store info in db if the chunk is the last

Parameters:chunk_info (dict) – Info about the chunk
Returns:local filename
Return type:str
store_file_info_in_db(name, filetype, file_name, size)

Store the file info in the database

Parameters:
  • name (string) – Name of the file
  • filetype (string) – Type (csv …)
  • file_name (string) – Local file name
  • size (string) – Size of file
write_data_into_file(data, file_name, mode)

Write data into a file

Parameters:
  • data (string) – data to write
  • file_name (string) – Local file name
  • mode (string) – open mode (w or a)
askomics.libaskomics.FilesUtils module
class askomics.libaskomics.FilesUtils.FilesUtils(app, session)

Bases: askomics.libaskomics.Params.Params

Contain methods usefull in FilesHandler and ResultsdHandler

get_size_occupied_by_user()

Get disk size occuped by file user (uploaded files and results)

Returns:size un bytes
Return type:int
askomics.libaskomics.Galaxy module
class askomics.libaskomics.Galaxy.Galaxy(app, session, url=None, apikey=None)

Bases: askomics.libaskomics.Params.Params

Connection with a Galaxy account

apikey

Galaxy API key

Type:string
url

Galaxy url

Type:string
check_galaxy_instance()

Check the Galaxy credentials

Returns:True if URL and Key exists
Return type:Boolean
download_datasets(datasets_id)

Download galaxy datasets into AskOmics

Parameters:datasets_id (list) – List of Galaxy datasets id
get_dataset_content(dataset_id)

Get Galaxy dataset content

Parameters:dataset_id (string) – dataset ID
Returns:Content of the dataset
Return type:string
get_datasets_and_histories(history_id=None, query=False)

Get Galaxy datasets of the current history and all histories

Parameters:
  • history_id (int, optional) – A history id
  • query (bool, optional) – Get Datasets, or json datasets for query
Returns:

Datasets and histories
Return type:

dict
askomics.libaskomics.GffFile module
class askomics.libaskomics.GffFile.GffFile(app, session, file_info, host_url=None, external_endpoint=None, custom_uri=None)

Bases: askomics.libaskomics.File.File

GFF File

public

Public or private dataset

Type:bool
generate_rdf_content()

Generator of the rdf content

Yields:Graph – Rdf content
get_preview()

Get gff file preview (list of entities)

Returns:Return info about the file
Return type:dict
integrate(entities, public=True)

Integrate GFF file

Parameters:
  • entities (List) – Entities to integrate
  • public (bool, optional) – Insert in public dataset
set_preview()

Summary

set_rdf_abstraction_domain_knowledge()

Set the abstraction and domain knowledge

askomics.libaskomics.LocalAuth module

Contain the Database class

class askomics.libaskomics.LocalAuth.LocalAuth(app, session)

Bases: askomics.libaskomics.Params.Params

Manage user authentication

add_galaxy_account(user, url, apikey)

Add a Galaxy account

Parameters:
  • user (dict) – Previous user info
  • url (string) – Galaxy URL
  • apikey (string) – Galaxy API key
Returns:

Updated user
Return type:

dict
authenticate_user(inputs)

check if the password is the good password associate with the email

Parameters:inputs (dict) – login and password
Returns:user info if authentication success
Return type:dict
authenticate_user_with_apikey(apikey)

Return the user associated with the API key

Parameters:inputs (string) – API key
Returns:user info if authentication success
Return type:dict
check_inputs(inputs)

Check user inputs

Check if inputs are not empty, if passwords are identical, and if username and email are not already in the database

Parameters:inputs (dict) – User inputs
create_directory(directory_path)

Create a directory

Parameters:directory_path (string) – Path
create_user_directories(user_id, username)

Create the User directory

Parameters:
  • user_id (int) – User id
  • username (string) – username
get_all_users()

Get all user info

Returns:All user info
Return type:list
get_number_of_users()

get the number of users in the DB

Returns:Number of user in the Database
Return type:int
get_user(username)

Get a specific user by his username

Parameters:username (string) – User username
Returns:The corresponding user
Return type:dict
is_email_in_db(email)

Check if the email is present in the database

Parameters:email (str) – Email
Returns:True if the email exist
Return type:bool
is_username_in_db(username)

Check if the username is present in the database

Parameters:username (str) – Username
Returns:True if the user exist
Return type:bool
persist_user(inputs, ldap=False)

Persist user in the TS

Parameters:
  • inputs (dict) – User infos
  • ldap (bool, optional) – If True, user is ldap
Returns:

The user
Return type:

dict
set_admin(new_status, username)

Set a new admin status to a user

Parameters:
  • new_status (boolean) – True for an admin
  • username (string) – The concerned username
set_blocked(new_status, username)

Set a new blocked status to a user

Parameters:
  • new_status (boolean) – True for blocked
  • username (string) – The concerned username
set_quota(quota, username)

Set a new quota to a user

Parameters:
  • quota (int) – New quota
  • username (string) – The concerned username
update_apikey(user)

Create a new api key and store in the database

Parameters:user (dict) – The current user
Returns:error, error message and updated user
Return type:dict
update_galaxy_account(user, url, apikey)

Update a Galaxy account

Parameters:
  • user (dict) – Previous user info
  • url (string) – Galaxy URL
  • apikey (string) – Galaxy API key
Returns:

Updated user
Return type:

dict
update_password(inputs, user)

Update the password of a user

Parameters:
  • inputs (dict) – Curent password and the new one (and confirmation)
  • user (dict) – The current user
Returns:

error, error message and updated user
Return type:

dict
update_profile(inputs, user)

Update the profile of a user

Parameters:
  • inputs (dict) – fields to update
  • user (dict) – The current user
Returns:

error, error message and updated user
Return type:

dict
askomics.libaskomics.Params module

Contain the Params class

class askomics.libaskomics.Params.Params(app, session)

Bases: object

Mother of all libaskomics classes

app

flask app

log

flask logger

session

flask session

settings

askomics settings (from ini)

get_error()
get_error_message()
logged_user()

Check if a user is logged

Returns:True if a user is logged
Return type:bool
str_to_bool(bool_str)

Convert a true/false string to a boolan value

Parameters:bool_str (str) – boolean string
Returns:True or False
Return type:bool
askomics.libaskomics.PrefixManager module
class askomics.libaskomics.PrefixManager.PrefixManager(app, session)

Bases: askomics.libaskomics.Params.Params

Manage sparql prefixes

askomics_namespace

askomics namespace, from config file

Type:str
askomics_prefix

askomics prefix, from config file

Type:str
prefix

dict of all prefixes

Type:dict
get_prefix()

Get all prefixes

Returns:prefixes
Return type:str
askomics.libaskomics.RdfFile module
class askomics.libaskomics.RdfFile.RdfFile(app, session, file_info, host_url=None, external_endpoint=None, custom_uri=None)

Bases: askomics.libaskomics.File.File

RDF (turtle) File

public

Public or private dataset

Type:bool
get_preview()

Get a preview of the frist 100 lines of a ttl file

Returns:Description
Return type:TYPE
integrate(public=False)

Integrate the file into the triplestore

Parameters:public (bool, optional) – Integrate in private or public graph
set_preview()

Summary

askomics.libaskomics.RdfGraph module
class askomics.libaskomics.RdfGraph.RdfGraph(app, session)

Bases: askomics.libaskomics.Params.Params

rdflib.graph wrapper

askomics_namespace

AskOmics napespace

Type:Namespace
askomics_prefix

AskOmics prefix

Type:Namespace
graph

rdflib graph

Type:Graph
ntriple

Number of triple in the graph

Type:int
add(triple)

Add a triple into the rdf graph

Parameters:triple (tuple) – triple to add
bind(a, b)

Bind a namespace

Parameters:
  • a (string) – prefix
  • b (string) – namespace
get_triple()

Get all triple

merge(other_graph)

Merge a graph into this graph

Parameters:other_graph (RdfGraph) – The graph to merge
serialize(destination=None, format='xml', base=None, encoding=None, **args)

Serialize the graph into a file

Parameters:
  • format (string) – rdf syntaxe
  • encoding (string) – Encoding
  • destination (string) – File destination
askomics.libaskomics.Result module
class askomics.libaskomics.Result.Result(app, session, result_info, force_no_db=False)

Bases: askomics.libaskomics.Params.Params

Result represent a query result file

celery_id

Celery job id

Type:str
file_name

file name

Type:str
file_path

file path

Type:str
graph_state

The json query graph state

Type:dict
id

database id

Type:int
result_path

results directory path

Type:str

Clean a link by removing coordinates and other stuff

Parameters:link (dict) – A graph link
Returns:Cleaned link
Return type:dict
clean_node(node)

Clean a node by removing coordinates and other stuff

Parameters:node (dict) – A graph node
Returns:Cleaned node
Return type:dict
delete_db_entry()

Delete results from db

delete_file_from_filesystem()

Remove result file from filesystem

delete_result()

Remove results from db and filesystem

format_graph_state(d3_graph_state)

Format Graph state

Remove coordinates and other things

Parameters:d3_graph_state (dict) – The d3 graph state
Returns:formatted graph state
Return type:dict
get_dir_path()

Get directory path

Returns:directory path
Return type:str
get_file_name()

Get file name

Returns:file name
Return type:str
get_file_preview()

Get a preview of the results file

Returns:headers and preview
Return type:list, list
get_graph_state(formated=False)

Get get_graph_state

Returns:graph state
Return type:dict
get_sparql_query()

Get the sparql query if exists

Returns:The sparql query
Return type:string
publish_query(public)

Insert query id and desc in the published_query table

rollback()

Delete file

save_in_db()

Save results file info into the database

save_result_in_file(headers, results)

Save query results in a csv file

Parameters:
  • headers (list) – List of results headers
  • results (list) – Query results
Returns:

File size
Return type:

int
send2galaxy(file2send)

Send files to Galaxy

send_query_to_galaxy()

Send the json query to a galaxy dataset

send_result_to_galaxy()

Send a result file to Galaxy

set_celery_id(celery_id)

Set celery id

Parameters:celery_id (string) – The celery id
set_info_from_db_with_id()

Set result info from the db

update_celery(celery_id)

Update celery id of result in database

Parameters:celery_id (string) – DescriThe celery idption
update_db_status(status, size=None, update_celery=False, error=False, error_message=None, traceback=None)

Update status of results in db

Parameters:
  • error (bool, optional) – True if error during integration
  • error_message (bool, optional) – Error string if error is True
update_description(description)

Change the result description

update_public_status(public)

Change public status

Parameters:public (bool) – New public status
askomics.libaskomics.ResultsHandler module
class askomics.libaskomics.ResultsHandler.ResultsHandler(app, session)

Bases: askomics.libaskomics.Params.Params

Handle results

delete_results(files_id)

Delete files

Parameters:files_id (list) – list of file id to delete
Returns:list of remaining files
Return type:list
get_files_info()

Get files info of the user

Returns:list of file info
Return type:list
get_public_queries()

Get id and description of published queries

Returns:List of published queries (id and description)
Return type:List
askomics.libaskomics.SparqlQueryBuilder module
class askomics.libaskomics.SparqlQueryBuilder.SparqlQueryBuilder(app, session)

Bases: askomics.libaskomics.Params.Params

Format a sparql query

private_graphs

all user private graph

Type:list
public_graphs

all public graph

Type:list
build_query_from_json(json_query, preview=False, for_editor=False)

Build a sparql query for the json dict of the query builder

Parameters:json_query (dict) – The json query from the query builder
Returns:SPARQL query
Return type:str
format_endpoint_name(endpoint)

Replace local url by “local triplestore”

Parameters:endpoint (string) – The endpoint name
Returns:Formated endpoint name
Return type:string
format_graph_name(graph)

Format graph name by removing base graph and timestamp

Parameters:graph (string) – The graph name
Returns:Formated graph name
Return type:string
format_query(query, limit=30, replace_froms=True, federated=False)

Format the Sparql query

  • remove all FROM
  • add FROM <graph> (public graph and user graph)
  • set a limit if not (or if its to big)
Parameters:
  • query (string) – sparql query to format
  • limit (int, optional) – Description
Returns:

formatted sparql query
Return type:

string
format_sparql_variable(name)

Format a name into a sparql variable by remove spacial char and add a ?

Parameters:name (string) – name to convert
Returns:The corresponding sparql variable
Return type:string
get_checked_asked_graphs(asked_graphs)

Check if asked graphs are present in public and private graphs

Parameters:asked_graphs (list) – list of graphs asked by the user
Returns:list of graphs asked by the user, in the public and private graphs
Return type:list
get_default_query()

Get the default query

Returns:the default query
Return type:str
get_default_query_with_prefix()

Get default query with the prefixes

Returns:default query with prefixes
Return type:str
get_endpoints_string()

get endpoint strngs for the federated query engine

Returns:the endpoint string
Return type:string
get_federated_froms()

Get @from string fir the federated query engine

Returns:The from string
Return type:string
get_federated_froms_from_graphs(graphs)

Get @from string fir the federated query engine

Returns:The from string
Return type:string
get_federated_line()

Get federtated line

Returns:@federate <endpoint1> <endpoint1> …
Return type:string
get_froms()

Get FROM string

Returns:FROM string
Return type:string
get_froms_from_graphs(graphs)

Get FROM’s form a list of graphs

Parameters:graphs (list) – List of graphs
Returns:from string
Return type:str
get_graphs_and_endpoints(selected_graphs=None, selected_endpoints=None)

get graphs and endpoints (uri and names)

Returns:List of dict uri name
Return type:list
is_bnode(uri, entities)

Check if a node uri is a blank node

Parameters:
  • uri (string) – node uri
  • entities (list) – all the entities
Returns:

True if uri correspond to a blank node
Return type:

Bool
is_federated()

Return True if there is more than 1 endpoint

Returns:True or False
Return type:bool
prefix_query(query)

Add prefix and dedent a sparql query string

Parameters:query (string) – The sparql query
Returns:Formatted query
Return type:string
replace_froms()

True if not federated and endpoint is local

Returns:True or False
Return type:bool
set_endpoints(endpoints)

Set endpoints

Parameters:endpoints (list) – Endpoints
set_graphs(graphs)

Set graphs

Parameters:graphs (list) – graphs
set_graphs_and_endpoints(entities=None, graphs=None, endpoints=None)

Get all public and private graphs containing the given entities

Parameters:entities (list, optional) – list of entity uri
toggle_public(graph, public)

Change public status of data into the triplestore

Parameters:
  • graph (string) – Graph to update public status
  • public (string) – true or false (string)
triple_dict_to_string(triple_dict)

Convert a triple dict into a triple string

Parameters:triple_dict (dict) – The triple dict
Returns:The triple string
Return type:string
askomics.libaskomics.SparqlQueryLauncher module
class askomics.libaskomics.SparqlQueryLauncher.SparqlQueryLauncher(app, session, get_result_query=False, federated=False, endpoints=None)

Bases: askomics.libaskomics.Params.Params

endpoint

The triplestore endpoint

Type:SPARQLWrapper
query_time

Query execution time

Type:time
triplestore

triplesotre (virtuoso, fuseki …)

Type:string
drop_dataset(graph)

Drop the datasets of the triplestore and its metadata

Parameters:graph (string) – graph name to remove
execute_query(query)

Execute a sparql query

Parameters:query (string) – Query to perform
Returns:result
Return type:TYPE
get_triples_from_graph(graph)

Get triples from a rdflib graph

Parameters:graph (Graph) – rdf graph
Returns:ttl string
Return type:string
insert_data(ttl, graph, metadata=False)

Insert data into the triplesotre using INSERT

Parameters:
  • ttl (Graph) – rdflib graph
  • graph (string) – graph name
  • metadata (bool, optional) – metadatas?
Returns:

query result
Return type:

TYPE
insert_ttl_string(ttl_string, graph)

Insert ttl into the triplestore

Parameters:
  • ttl_string (string) – ttl triples to insert
  • graph (string) – Insert in the named graph
Returns:

query result
Return type:

dict?
load_data(file_name, graph, host_url)

Load data in function of the triplestore

Parameters:
  • file_name (string) – File name to load
  • graph (string) – graph name
  • host_url (string) – AskOmics url
load_data_fuseki(file_name, graph)

Load data using fuseki load request

Parameters:
  • file_name (string) – File name to load
  • graph (string) – graph name
Returns:

Response of request
Return type:

response
load_data_virtuoso(file_name, graph, host_url)

Load data using virtuoso load query

Parameters:
  • file_name (string) – File name to load
  • graph (string) – graph name
  • host_url (string) – AskOmics url
Returns:

result of query
Return type:

TYPE
parse_results(json_results)

Parse result of sparql query

Parameters:json_results (dict) – Query result
Returns:Header and data
Return type:list, list
parse_results_old(json_results)

Parse result of sparql query

Parameters:json_results – Result of the query
Returns:parsed results
Return type:list
process_query(query)

Execute a query and return parsed results

Parameters:query (string) – The query to execute
Returns:Parsed results
Return type:list
askomics.libaskomics.Start module

Contain the Start classe

class askomics.libaskomics.Start.Start(app, session)

Bases: askomics.libaskomics.Params.Params

Initialize the data directory and the database

data_directory

Path to the data directory

Type:str
database_path

Path to the database file

Type:str
create_data_directory()

Create the data directory if it not exists

create_database()

Initialize the database file

start()

Create the data diretory and initialize the database file

askomics.libaskomics.TriplestoreExplorer module
class askomics.libaskomics.TriplestoreExplorer.TriplestoreExplorer(app, session)

Bases: askomics.libaskomics.Params.Params

Explore the triplestore

check_presence(uri, list_of_things)

Check if an uri is present in a list of dict

Parameters:
  • uri (string) – the uri to test
  • list_of_things (list) – the list of dict[‘uri’]
Returns:

True if the uri is present
Return type:

bool
get_abstraction()

Get AskOmics Abstraction

Returns:AskOmics abstraction
Return type:dict
get_abstraction_attributes()

Get user abstraction attributes from the triplestore

Returns:AskOmics attributes
Return type:list
get_abstraction_entities()

Get abstraction entities

Returns:List of entities available
Return type:list
get_abstraction_relations()

Get user abstraction relations from the triplestore

Returns:Relations
Return type:list
get_attribute_index(uri, attribute_list)

Get attribute index

Parameters:
  • uri (string) – uri of the attribute
  • attribute_list (list) – list of attributes
Returns:

Index of the given uri in the list
Return type:

int
get_startpoints()

Get public and user startpoints

Returns:Startpoints
Return type:list
askomics.libaskomics.Utils module
class askomics.libaskomics.Utils.Utils

Bases: object

Contain utils fonction and classes

static get_random_string(number)

return a random string of n character

Parameters:number (int) – number of character of the random string
Returns:a random string of n chars
Return type:str
static humansize_to_bytes(hsize)

Convert human-readable string into bytes

Parameters:hsize (string) – Human readable string
Returns:Bytes
Return type:int
static intersect(a, b)

return the intersection of two lists

static is_url(url)

Check is string is an url

Parameters:url (string) – string to test
Returns:True if string is url
Return type:bool
static is_valid_url(url)

Test if a string an url

Parameters:url (string) – The url to test
Returns:True is url is valid
Return type:bool
static union(a, b)

return the union of two lists

static unique(l)

return the list with duplicate elements removed and keep order

class askomics.libaskomics.Utils.cached_property(func)

Bases: object

Like @property on a member function, but also cache the calculation in self.__dict__[function name]. The function is called only once since the cache stored as an instance attribute override the property residing in the class attributes. Following accesses cost no more than standard Python attribute access. If the instance attribute is deleted the next access will re-evaluate the function. Source: https://blog.ionelmc.ro/2014/11/04/an-interesting-python-descriptor-quirk/

class Shape(object):

@cached_property def area(self):

# compute value return value
func

Description

Type:TYPE
func
Module contents

Submodules

askomics.app module

AskOmics app

askomics.app.BLUEPRINTS

Flask blueprints

Type:Tuple
askomics.app.create_app(config='config/askomics.ini', app_name='askomics', blueprints=None)

Create the AskOmics app

Parameters:
  • config (str, optional) – Path to the config file
  • app_name (str, optional) – Application name
  • blueprints (None, optional) – Flask blueprints
Returns:

AskOmics Flask application
Return type:

Flask
askomics.app.create_celery(app)

Create the celery object

Parameters:app (Flask) – AskOmics Flask application
Returns:Celery object
Return type:Celery

askomics.tasks module

Module contents

Indices and tables