sslhep/ml_front

Sponsored OSS

By University of Chicago

Updated about 4 years ago

ML services on K8s

Image
Languages & frameworks
Machine learning & AI
Data science
0

10K+

sslhep/ml_front repository overview

ML front

Exposes k8s ml resources to users as services:

  • JupyterLab.
  • TensorFlowAsAService (soon)

A deployment creates node.js based web frontend. The front end authenticates users through globus. User can select to instantiate several different services, select hardware needed, duration, and other parameters.

Configuration options

VariableMeaningExample value
EVENTThis will be used in mails, kibana dashboardscodas2019
SITENAMEaddress where the front end will be accessible fromml.maniac.uchicago.edu
STATIC_PATHdirectory containing static site customizationsml-usatlas-org,
NAMESPACEk8s namespace where all of pods and services will runmaniac-ml
TITLEWill be shown in web browser title bar
SSLAre JupyterLabs served on https?false
APPROVAL_REQUIREDIs authorization required (on top of globus authentication)?true
APPROVAL_EMAILOnly if authorization is required[email protected]
SINGLE_INSTANCELimit to only one private Jupyter instancefalse
PUBLIC_INSTANCEExpose a public JupyterLab instance that anyone can usetrue
MONITOREnable user access to monitoring infotrue
REMOTE_K8SEnable non local k8s service spawnfalse
REPORTINGAre pods monitored in Elasticsearctrue
JL_PODUsed to customize JuputerLab pod./jupyter-pod.json
JL_SERVICEUsed to customize JuputerLab service./jupyter-service.json
PLUGINSlist of enabled plugins["MONITOR", "SPARK", "PUBLIC_INSTANCE"]

Authorizing users

If configured to require authorization (APPROVAL_REQUIRED), authorization is automatically sought for authenticated users. Authorization request is approved by a platform owner.

To look up users check kibana page:

http://atlas-kibana.mwt2.org/goto/7ad9cbf9627d180d24e06ac018ac6c4a

TODO

  • switch from using .json files to fully generated objects.
  • completely move to ingress controllers. (spark too)
  • change how upper left corner Title is set (should be in pug.)
  • add shared filesystem
  • add option to mount CVMFS
  • add option to update some service parameters
  • add cloud deployments
  • add postfixes to pods/services
  • add ml-front pods monitoring info collection
  • SPARK submission
    • submission page
      • file upload (multiple files?)
      • n executors
      • executor memory
      • job name
    • results page. status, link to output.
  • write documentation
  • TFAAS
  • web site
    • Services - running/terminated service
    • Services - services title

Tag summary

Content type

Image

Digest

Size

379.3 MB

Last updated

about 4 years ago

docker pull sslhep/ml_front

This week's pulls

Pulls:

69

Jun 29 to Jul 5