Katib

Reference documentation for Katib

Table of Contents

Top

api.proto

Katib GRPC API v1beta1

AlgorithmSetting

HP or NAS algorithm settings.

Field Type Label Description
name string
value string

AlgorithmSpec

HP or NAS algorithm specification.

Field Type Label Description
algorithm_name string
algorithm_settings AlgorithmSetting repeated

DeleteObservationLogReply

DeleteObservationLogRequest

Field Type Label Description
trial_name string

EarlyStoppingRule

EarlyStoppingRule represents single early stopping rule.

Field Type Label Description
name string Name of the rule. Usually, metric name.
value string Value of the metric.
comparison ComparisonType Correlation between name and value, one of equal, less or greater
start_step int32 Defines quantity of intermediate results that should be received before applying the rule. If start step is empty, rule is applied from the first recorded metric.

EarlyStoppingSetting

Early stopping algorithm settings.

Field Type Label Description
name string
value string

EarlyStoppingSpec

Early stopping algorithm specification.

Field Type Label Description
algorithm_name string
algorithm_settings EarlyStoppingSetting repeated

Experiment

Structure for a single Experiment.

Field Type Label Description
name string Name for the Experiment.
spec ExperimentSpec Experiment specification.

ExperimentSpec

Specification of an Experiment. Experiment represents a single optimization run over a feasible space. Each Experiment contains a configuration describing the feasible space, as well as a set of Trials. It is assumed that objective function f(x) does not change in the course of an Experiment.

Field Type Label Description
parameter_specs ExperimentSpec.ParameterSpecs
objective ObjectiveSpec Objective specification for the Experiment.
algorithm AlgorithmSpec HP or NAS algorithm specification for the Experiment.
early_stopping EarlyStoppingSpec Early stopping specification for the Experiment.
parallel_trial_count int32 How many Trials can be processed in parallel.
max_trial_count int32 Max completed Trials to mark Experiment as succeeded.
nas_config NasConfig NAS configuration for the Experiment.

ExperimentSpec.ParameterSpecs

List of ParameterSpec.

Field Type Label Description
parameters ParameterSpec repeated

FeasibleSpace

Feasible space for optimization. Int and Double type use Max/Min. Discrete and Categorical type use List.

Field Type Label Description
max string Max Value
min string Minimum Value
list string repeated List of Values.
step string Step for double or int parameter

GetEarlyStoppingRulesReply

Field Type Label Description
early_stopping_rules EarlyStoppingRule repeated

GetEarlyStoppingRulesRequest

Field Type Label Description
experiment Experiment
trials Trial repeated
db_manager_address string

GetObservationLogReply

Field Type Label Description
observation_log ObservationLog

GetObservationLogRequest

Field Type Label Description
trial_name string
metric_name string
start_time string The start of the time range. RFC3339 format
end_time string The end of the time range. RFC3339 format

GetSuggestionsReply

Field Type Label Description
parameter_assignments GetSuggestionsReply.ParameterAssignments repeated
algorithm AlgorithmSpec
early_stopping_rules EarlyStoppingRule repeated

GetSuggestionsReply.ParameterAssignments

Field Type Label Description
assignments ParameterAssignment repeated

GetSuggestionsRequest

Field Type Label Description
experiment Experiment
trials Trial repeated all completed trials owned by the experiment.
request_number int32 The number of Suggestion you request at one time. When you set 3 to request_number, you can get three Suggestions at one time.

GraphConfig

GraphConfig contains a config of DAG

Field Type Label Description
num_layers int32 Number of layers
input_sizes int32 repeated Dimensions of input size
output_sizes int32 repeated Dimensions of output size

Metric

Field Type Label Description
name string
value string

MetricLog

Field Type Label Description
time_stamp string RFC3339 format
metric Metric

NasConfig

NasConfig contains a config of NAS job

Field Type Label Description
graph_config GraphConfig Config of DAG
operations NasConfig.Operations List of Operation

NasConfig.Operations

Field Type Label Description
operation Operation repeated

ObjectiveSpec

Objective specification.

Field Type Label Description
type ObjectiveType Type of optimization.
goal double Goal of optimization, can be empty.
objective_metric_name string Primary metric name for the optimization.
additional_metric_names string repeated List of additional metrics to record from Trial. This can be empty if we only care about the objective metric.

Observation

Field Type Label Description
metrics Metric repeated

ObservationLog

Field Type Label Description
metric_logs MetricLog repeated

Operation

Config for operations in DAG

Field Type Label Description
operation_type string Type of operation in DAG
parameter_specs Operation.ParameterSpecs

Operation.ParameterSpecs

List of ParameterSpec

Field Type Label Description
parameters ParameterSpec repeated

ParameterAssignment

Field Type Label Description
name string
value string

ParameterSpec

Config for a hyperparameter. Katib will create each Hyper parameter from this config.

Field Type Label Description
name string Name of the parameter.
parameter_type ParameterType Type of the parameter.
feasible_space FeasibleSpace FeasibleSpace for the parameter.

ReportObservationLogReply

ReportObservationLogRequest

Field Type Label Description
trial_name string
observation_log ObservationLog

SetTrialStatusReply

SetTrialStatusRequest

Field Type Label Description
trial_name string

Trial

Structure for a single Trial.

Field Type Label Description
name string Name for the Trial.
spec TrialSpec Trial specification.
status TrialStatus Trial status.

TrialSpec

Specification of a Trial. It represents Trial's parameter assignments and objective.

Field Type Label Description
objective ObjectiveSpec Objective specification for the Trial.
parameter_assignments TrialSpec.ParameterAssignments List of assignments generated for the Trial.

TrialSpec.ParameterAssignments

List of ParameterAssignment

Field Type Label Description
assignments ParameterAssignment repeated

TrialStatus

Current Trial status. It contains Trial's latest condition, start time, completion time, observation.

Field Type Label Description
start_time string Trial start time in RFC3339 format
completion_time string Trial completion time in RFC3339 format
condition TrialStatus.TrialConditionType Trial current condition. It is equal to the latest Trial CR condition.
observation Observation The best Trial observation in logs.

ValidateAlgorithmSettingsReply

Return INVALID_ARGUMENT Error if Algorithm Settings are not Valid

ValidateAlgorithmSettingsRequest

Field Type Label Description
experiment Experiment

ComparisonType

Name Number Description
UNKNOWN_COMPARISON 0 Unknown comparison, not used
EQUAL 1 Equal comparison, e.g. accuracy = 0.7
LESS 2 Less comparison, e.g. accuracy < 0.7
GREATER 3 Greater comparison, e.g. accuracy > 0.7

ObjectiveType

Direction of optimization. Minimize or Maximize.

Name Number Description
UNKNOWN 0 Undefined type and not used.
MINIMIZE 1 Minimize
MAXIMIZE 2 Maximize

ParameterType

Types of value for HyperParameter.

Name Number Description
UNKNOWN_TYPE 0 Undefined type and not used.
DOUBLE 1 Double float type. Use "Max/Min".
INT 2 Int type. Use "Max/Min".
DISCRETE 3 Discrete number type. Use "List" as float.
CATEGORICAL 4 Categorical type. Use "List" as string.

TrialStatus.TrialConditionType

Trial can be in one of 6 conditions. TODO (andreyvelich): Remove unused conditions.

Name Number Description
CREATED 0
RUNNING 1
SUCCEEDED 2
KILLED 3
FAILED 4
EARLYSTOPPED 5
UNKNOWN 6

DBManager

DBManager service defines APIs to manage Katib database.

Method Name Request Type Response Type Description
ReportObservationLog ReportObservationLogRequest ReportObservationLogReply Report a log of Observations for a Trial. The log consists of timestamp and value of metric. Katib store every log of metrics. You can see accuracy curve or other metric logs on UI.
GetObservationLog GetObservationLogRequest GetObservationLogReply Get all log of Observations for a Trial.
DeleteObservationLog DeleteObservationLogRequest DeleteObservationLogReply Delete all log of Observations for a Trial.

EarlyStopping

EarlyStopping service defines APIs to manage Katib Early Stopping algorithms

Method Name Request Type Response Type Description
GetEarlyStoppingRules GetEarlyStoppingRulesRequest GetEarlyStoppingRulesReply
SetTrialStatus SetTrialStatusRequest SetTrialStatusReply

Suggestion

Suggestion service defines APIs to manage Katib Suggestion from HP or NAS algorithms

Method Name Request Type Response Type Description
GetSuggestions GetSuggestionsRequest GetSuggestionsReply
ValidateAlgorithmSettings ValidateAlgorithmSettingsRequest ValidateAlgorithmSettingsReply

Scalar Value Types

.proto Type Notes C++ Java Python Go C# PHP Ruby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)