Vantage6 Community Meeting Highlights - September 21, 2023
Infrastructure and algorithm developers, committers, and contributors came together at the Vantage6 Community Meeting on September 21, 2023. The meeting focused on collaboration opportunities, recent releases, and upcoming events. Participants discussed important projects like the AdVantage healthcare initiative and shared development efforts to align goals. Bug reports were encouraged to be submitted via the Github issue tracker for efficient resolution.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Community meeting 21thof September 2023
The community Infrastructure Developers Committers & contributors Algorithm Developers Creating federated analysis algorithms Users Use vantage6 for privacy preserving analysis
Aim Collaboration opportunities Finding common interests by sharing what you are working on Align development efforts Avoid doing things twice Bug reports Please submit these in our issue tracker: https://github.com/vantage6/vantage6/issues
Agenda 11:00 Short introduction 11:05 Announcements and upcoming events 11:10 Recently released 11:25 Infrastructure development 11:40 Algorithm development 11:55 Miscellaneous
Announcements and upcoming events 12th Oct 2023 @Utrecht Health-RI (link) come visit the vantage6 booth! say hi! DataSHIELD Conference (link) 11-13th Oct 2023 @Groningen
AdVantage project healthcare vantage6 Plan: to develop "something" that will benefit the v6 community. This week we kicked off with a requirements session with people from the v6 community and a kickoff with Frank and Bart eScience Contributors: Djura Smits, Cunliang Geng, Walter Baccinelli, Hector Cavadid IKNL Contributors: Frank Martin, Bart van Beusekom Timeframe: We have approx. 5000 hours to spend between now and 2025.
Recently released 3.11.0 4.0.0 (3 releases)
Feature: Automate test setup With a single command, create a server, a few nodes, and a collaboration in order to test the vantage6 infrastructure. Contributors: Hasan Alradhi, Frank Martin, Bart van Beusekom [Feature Request] Create demo collaboration
Feature: Start User Interface from the CLI There is a docker image available for the UI, but it could not be started together with the server. This is now simplified. Contributors: Bart van Beusekom, Frank Martin [Feature Request] Start UI locally with vserver
Version 4.0: Cotopaxi Several breaking changes have been made to improve further development. This includes improving the performance of the server (pagination, API output), improved algorithm dev and a new API as alternative to file mounts. Contributors: Bart van Beusekom, IKNL; Frank Martin, IKNL Project board 4.0.0
[Feature] Algorithm decorators (4.0) You can decorate algorithm functions to indicate they get an AlgorithmClient or one or more datasets. This removes the distinction between the master functions and RPC functions we had until now. Contributors: Bart van Beusekom, IKNL; Frank Martin, IKNL Estimate:Q3 2023 [Change] Algorithm decorators
[Feature] Collaboration scope (4.0) In the past, permissions were given at organization or at global level. Now there is a full in-between collaboration scope. Contributors: Bart van Beusekom, IKNL; Frank Martin, IKNL [Feature Request] Extend API security layers with Collaboration scope
[Feature] API performance (4.0) Vantage6 API requests will be paginated by default. User Interface should be much faster as tables in there will be paginated as well. Additionally, what is returned is shorter. For instance, requesting an organization will no longer give a link for each task, but one link to get all tasks of that organization. Contributors: Bart van Beusekom, IKNL; Frank Martin, IKNL [Feature Request] Extend API to enable pagination in UI
[Change] Remove environments in config files (4.0) One could specify environments in node/server configuration files. In practice, these were very rarely used (it was easier to create a separate dev configuration). The environments will therefore be removed. Contributors: Bart van Beusekom, IKNL; Frank Martin, IKNL [Feature Request] Get rid of environments in config files
[Change] Rename result to run (4.0) client.result.get() When you run an algorithm container, the data was stored in result . This leads to confusing fields such as result.input, result.started_at and of course result.result. client.run.get() AND client.result.get() The name run makes more sense here. Contributors: Bart van Beusekom, IKNL; Frank Martin, IKNL [Feature Request] Get rid of environments in config files
[Change] Split vantage6-client into two packages (4.0) The Python client package also contains the algorithm (wrapper) tools. These may be separated in a different package. This would also reduce dependencies required for the client. Contributors: Bart van Beusekom, IKNL; Frank Martin, IKNL [Change Request] Split vantage6-client into two packages
[Change] Share node database labels and types with server Mostly for UI development. We want to be able to verify that nodes have defined the same database labels. Knowing type is useful for e.g. providing queries in case of SQL databases. Contributors: Bart van Beusekom, IKNL; Frank Martin, IKNL [Feature Request] Share a node's database labels and types with central server
[Security] Using JSON instead of pickle for serialization Serialization and deserialization of algorithm results was done with pickle by default for algorithms using the Python wrappers. Due to security concerns around pickles, JSON will be the default and pickles will no longer be supported. Contributors: Bart van Beusekom, IKNL; Frank Martin, IKNL [Security] Removing pickle serialization/deserialization
Other changes 4 14 changes 14 features 5 bugfixes security patches For more details on minor features and resolved bugs, see https://docs.vantage6.ai/en/main/release_notes.html
Infrastructure development Upcoming features and changes
[Feature] OMOP support Connect algorithms to an OMOP data- Option 1) vantage6 SQL interface source. The first step is to extract a default data set from OMOP, in a later stage the user should be able to specify the data to be extracted. Option 2) vantage6 HTTP interface Contributors: Frank Martin, IKNL; Estimate: Q4 2023 https://python-ohdsi.readthedocs.io/
[Feature] Algorithm preprocessing For example: Data transformation Feature Engineering Stratification Discretization Filtering Contributors: Harm Buisman, Frank Martin, Bart van Beusekom (IKNL) Estimate: Q4 2023
Feature: Researcher UI The User Interface is currently basically a management tool around the API. We would like to extend it to include a researcher perspective that facilitates sending tasks and showing graphs based on algorithm metadata. Contributors: Richard Leurs, Elastique; Bart van Beusekom, IKNL; Frank Martin, IKNL Estimate:Q4 2023
Feature: Algorithm Store It is difficult to get an overview of existing vantage6 algorithms, and except for the docker image not much information is available. The algorithm store will be a new User Interface that lists available algorithms for your project. The store will also establish trust in algorithms because it will be clear who is developer / reviewer / owner etc. Contributors: Bart van Beusekom, IKNL; Frank Martin, IKNL; Ananya Choudhury, Maastro Estimate:2024
[Feature] Flexible algorithm template The algorithm boilerplate that we have at present is not flexible, difficult to keep up- to-date and not everyone needs all code. We want to generate personalized templates for algorithm developers. They answer questions about e.g. if the algorithm should use VPN and examples will be created for that. Contributors: Bart van Beusekom, Frank Martin (IKNL) Estimate: Q4 2023
Security and privacy - Cox ZPH We are preparing a document to describe the risks of a particular algorithm, which should also serve as a blueprint to describe risks for other algorithms. Contributors: Hasan Alradhi2, Matteo Cellemare1, Frank Martin2, Bart van Beusekom2 Estimate:Completed (Algorithm & Security and privacy doc) 1: Georgetown University, 2: IKNL
Survival analysis algorithms: Kaplan Meier Horizontally partitioned Kaplan-Meier with a plot and a separate contingency table algorithm. We will also be writing a Security and privacy document about each of these algorithms. Contributors: Hasan Alradhi2, Matteo Cellemare1 Estimate:September 2023 (Algorithm), Security and privacy end of year. 1: Georgetown University, 2: IKNL
Cross Tabulation Horizontally Federated Cross Tabulation algorithm (Contingency table) while hiding sensitive frequency information. We will also be writing a security and privacy document for this. Contributors: Hasan Alradhi2, Matteo Cellemare1 Estimate:Completed (Algorithm), Security and privacy by end of year. 1: Georgetown University, 2: IKNL
GLM, GLMM and Hierarchical models Part of Conor s PhD papers develop multiple statistical models to answer interesting clinical questions in a federated setting. Contributors: Hasan Alradhi2, Conor Hassan1 Estimate:July 2024 (Phd deadline) 1: QUT, Cancer Council Queensland, 2: IKNL
Synthetic Data Synthesize the IKNL core dataset (collection of Cancer data). Contributors: Chiara Attanasio1, Hasan Alradhi1, Estimate:Ongoing 1:IKNL
Miscellaneous Questions Finding common interests by sharing what you are working on Feedback Finding common interests by sharing what you are working on Ideas Finding common interests by sharing what you are working on
