Katib

Table of Contents

• api.proto

  • AlgorithmSetting

  • AlgorithmSpec

  • DeleteObservationLogReply

  • DeleteObservationLogRequest

  • EarlyStoppingRule

  • EarlyStoppingSetting

  • EarlyStoppingSpec

  • Experiment

  • ExperimentSpec

  • ExperimentSpec.ParameterSpecs

  • FeasibleSpace

  • GetEarlyStoppingRulesReply

  • GetEarlyStoppingRulesRequest

  • GetObservationLogReply

  • GetObservationLogRequest

  • GetSuggestionsReply

  • GetSuggestionsReply.ParameterAssignments

  • GetSuggestionsRequest

  • GraphConfig

  • Metric

  • MetricLog

  • NasConfig

  • NasConfig.Operations

  • ObjectiveSpec

  • Observation

  • ObservationLog

  • Operation

  • Operation.ParameterSpecs

  • ParameterAssignment

  • ParameterSpec

  • ReportObservationLogReply

  • ReportObservationLogRequest

  • SetTrialStatusReply

  • SetTrialStatusRequest

  • Trial

  • TrialSpec

  • TrialSpec.ParameterAssignments

  • TrialStatus

  • ValidateAlgorithmSettingsReply

  • ValidateAlgorithmSettingsRequest

  • ComparisonType

  • ObjectiveType

  • ParameterType

  • TrialStatus.TrialConditionType

  • DBManager

  • EarlyStopping

  • Suggestion

• Scalar Value Types

Top

api.proto

Katib GRPC API v1beta1

AlgorithmSetting

HP or NAS algorithm settings.

Field	Type	Label	Description
name	string
value	string

AlgorithmSpec

HP or NAS algorithm specification.

Field	Type	Label	Description
algorithm_name	string
algorithm_settings	AlgorithmSetting	repeated

DeleteObservationLogReply

DeleteObservationLogRequest

Field	Type	Label	Description
trial_name	string

EarlyStoppingRule

EarlyStoppingRule represents single early stopping rule.

Field	Type	Label	Description
name	string		Name of the rule. Usually, metric name.
value	string		Value of the metric.
comparison	ComparisonType		Correlation between name and value, one of equal, less or greater
start_step	int32		Defines quantity of intermediate results that should be received before applying the rule. If start step is empty, rule is applied from the first recorded metric.

EarlyStoppingSetting

Early stopping algorithm settings.

Field	Type	Label	Description
name	string
value	string

EarlyStoppingSpec

Early stopping algorithm specification.

Field	Type	Label	Description
algorithm_name	string
algorithm_settings	EarlyStoppingSetting	repeated

Experiment

Structure for a single Experiment.

Field	Type	Label	Description
name	string		Name for the Experiment.
spec	ExperimentSpec		Experiment specification.

ExperimentSpec

Specification of an Experiment. Experiment represents a single optimization run over a feasible space. Each Experiment contains a configuration describing the feasible space, as well as a set of Trials. It is assumed that objective function f(x) does not change in the course of an Experiment.

Field	Type	Label	Description
parameter_specs	ExperimentSpec.ParameterSpecs
objective	ObjectiveSpec		Objective specification for the Experiment.
algorithm	AlgorithmSpec		HP or NAS algorithm specification for the Experiment.
early_stopping	EarlyStoppingSpec		Early stopping specification for the Experiment.
parallel_trial_count	int32		How many Trials can be processed in parallel.
max_trial_count	int32		Max completed Trials to mark Experiment as succeeded.
nas_config	NasConfig		NAS configuration for the Experiment.

ExperimentSpec.ParameterSpecs

List of ParameterSpec.

Field	Type	Label	Description
parameters	ParameterSpec	repeated

FeasibleSpace

Feasible space for optimization. Int and Double type use Max/Min. Discrete and Categorical type use List.

Field	Type	Label	Description
max	string		Max Value
min	string		Minimum Value
list	string	repeated	List of Values.
step	string		Step for double or int parameter

GetEarlyStoppingRulesReply

Field	Type	Label	Description
early_stopping_rules	EarlyStoppingRule	repeated

GetEarlyStoppingRulesRequest

Field	Type	Label	Description
experiment	Experiment
trials	Trial	repeated
db_manager_address	string

GetObservationLogReply

Field	Type	Label	Description
observation_log	ObservationLog

GetObservationLogRequest

Field	Type	Label	Description
trial_name	string
metric_name	string
start_time	string		The start of the time range. RFC3339 format
end_time	string		The end of the time range. RFC3339 format

GetSuggestionsReply

Field	Type	Label	Description
parameter_assignments	GetSuggestionsReply.ParameterAssignments	repeated
algorithm	AlgorithmSpec
early_stopping_rules	EarlyStoppingRule	repeated

GetSuggestionsReply.ParameterAssignments

Field	Type	Label	Description
assignments	ParameterAssignment	repeated

GetSuggestionsRequest

Field	Type	Label	Description
experiment	Experiment
trials	Trial	repeated	all completed trials owned by the experiment.
request_number	int32		The number of Suggestion you request at one time. When you set 3 to request_number, you can get three Suggestions at one time.

GraphConfig

GraphConfig contains a config of DAG

Field	Type	Label	Description
num_layers	int32		Number of layers
input_sizes	int32	repeated	Dimensions of input size
output_sizes	int32	repeated	Dimensions of output size

Metric

Field	Type	Label	Description
name	string
value	string

MetricLog

Field	Type	Label	Description
time_stamp	string		RFC3339 format
metric	Metric

NasConfig

NasConfig contains a config of NAS job

Field	Type	Label	Description
graph_config	GraphConfig		Config of DAG
operations	NasConfig.Operations		List of Operation

NasConfig.Operations

Field	Type	Label	Description
operation	Operation	repeated

ObjectiveSpec

Objective specification.

Field	Type	Label	Description
type	ObjectiveType		Type of optimization.
goal	double		Goal of optimization, can be empty.
objective_metric_name	string		Primary metric name for the optimization.
additional_metric_names	string	repeated	List of additional metrics to record from Trial. This can be empty if we only care about the objective metric.

Observation

Field	Type	Label	Description
metrics	Metric	repeated

ObservationLog

Field	Type	Label	Description
metric_logs	MetricLog	repeated

Operation

Config for operations in DAG

Field	Type	Label	Description
operation_type	string		Type of operation in DAG
parameter_specs	Operation.ParameterSpecs

Operation.ParameterSpecs

List of ParameterSpec

Field	Type	Label	Description
parameters	ParameterSpec	repeated

ParameterAssignment

Field	Type	Label	Description
name	string
value	string

ParameterSpec

Config for a hyperparameter. Katib will create each Hyper parameter from this config.

Field	Type	Label	Description
name	string		Name of the parameter.
parameter_type	ParameterType		Type of the parameter.
feasible_space	FeasibleSpace		FeasibleSpace for the parameter.

ReportObservationLogReply

ReportObservationLogRequest

Field	Type	Label	Description
trial_name	string
observation_log	ObservationLog

SetTrialStatusReply

SetTrialStatusRequest

Field	Type	Label	Description
trial_name	string

Trial

Structure for a single Trial.

Field	Type	Label	Description
name	string		Name for the Trial.
spec	TrialSpec		Trial specification.
status	TrialStatus		Trial status.

TrialSpec

Specification of a Trial. It represents Trial's parameter assignments and objective.

Field	Type	Label	Description
objective	ObjectiveSpec		Objective specification for the Trial.
parameter_assignments	TrialSpec.ParameterAssignments		List of assignments generated for the Trial.

TrialSpec.ParameterAssignments

List of ParameterAssignment

Field	Type	Label	Description
assignments	ParameterAssignment	repeated

TrialStatus

Current Trial status. It contains Trial's latest condition, start time, completion time, observation.

Field	Type	Label	Description
start_time	string		Trial start time in RFC3339 format
completion_time	string		Trial completion time in RFC3339 format
condition	TrialStatus.TrialConditionType		Trial current condition. It is equal to the latest Trial CR condition.
observation	Observation		The best Trial observation in logs.

ValidateAlgorithmSettingsReply

Return INVALID_ARGUMENT Error if Algorithm Settings are not Valid

ValidateAlgorithmSettingsRequest

Field	Type	Label	Description
experiment	Experiment

ComparisonType

Name	Number	Description
UNKNOWN_COMPARISON	0	Unknown comparison, not used
EQUAL	1	Equal comparison, e.g. accuracy = 0.7
LESS	2	Less comparison, e.g. accuracy < 0.7
GREATER	3	Greater comparison, e.g. accuracy > 0.7

ObjectiveType

Direction of optimization. Minimize or Maximize.

Name	Number	Description
UNKNOWN	0	Undefined type and not used.
MINIMIZE	1	Minimize
MAXIMIZE	2	Maximize

ParameterType

Types of value for HyperParameter.

Name	Number	Description
UNKNOWN_TYPE	0	Undefined type and not used.
DOUBLE	1	Double float type. Use "Max/Min".
INT	2	Int type. Use "Max/Min".
DISCRETE	3	Discrete number type. Use "List" as float.
CATEGORICAL	4	Categorical type. Use "List" as string.

TrialStatus.TrialConditionType

Trial can be in one of 6 conditions. TODO (andreyvelich): Remove unused conditions.

Name	Number	Description
CREATED	0
RUNNING	1
SUCCEEDED	2
KILLED	3
FAILED	4
EARLYSTOPPED	5
UNKNOWN	6

DBManager

DBManager service defines APIs to manage Katib database.

Method Name	Request Type	Response Type	Description
ReportObservationLog	ReportObservationLogRequest	ReportObservationLogReply	Report a log of Observations for a Trial. The log consists of timestamp and value of metric. Katib store every log of metrics. You can see accuracy curve or other metric logs on UI.
GetObservationLog	GetObservationLogRequest	GetObservationLogReply	Get all log of Observations for a Trial.
DeleteObservationLog	DeleteObservationLogRequest	DeleteObservationLogReply	Delete all log of Observations for a Trial.

EarlyStopping

EarlyStopping service defines APIs to manage Katib Early Stopping algorithms

Method Name	Request Type	Response Type	Description
GetEarlyStoppingRules	GetEarlyStoppingRulesRequest	GetEarlyStoppingRulesReply
SetTrialStatus	SetTrialStatusRequest	SetTrialStatusReply

Suggestion

Suggestion service defines APIs to manage Katib Suggestion from HP or NAS algorithms

Method Name	Request Type	Response Type	Description
GetSuggestions	GetSuggestionsRequest	GetSuggestionsReply
ValidateAlgorithmSettings	ValidateAlgorithmSettingsRequest	ValidateAlgorithmSettingsReply

Scalar Value Types

.proto Type	Notes	C++	Java	Python	Go	C#	PHP	Ruby
double		double	double	float	float64	double	float	Float
float		float	float	float	float32	float	float	Float
int32	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
int64	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.	int64	long	int/long	int64	long	integer/string	Bignum
uint32	Uses variable-length encoding.	uint32	int	int/long	uint32	uint	integer	Bignum or Fixnum (as required)
uint64	Uses variable-length encoding.	uint64	long	int/long	uint64	ulong	integer/string	Bignum or Fixnum (as required)
sint32	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sint64	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.	int64	long	int/long	int64	long	integer/string	Bignum
fixed32	Always four bytes. More efficient than uint32 if values are often greater than 2^28.	uint32	int	int	uint32	uint	integer	Bignum or Fixnum (as required)
fixed64	Always eight bytes. More efficient than uint64 if values are often greater than 2^56.	uint64	long	int/long	uint64	ulong	integer/string	Bignum
sfixed32	Always four bytes.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sfixed64	Always eight bytes.	int64	long	int/long	int64	long	integer/string	Bignum
bool		bool	boolean	boolean	bool	bool	boolean	TrueClass/FalseClass
string	A string must always contain UTF-8 encoded or 7-bit ASCII text.	string	String	str/unicode	string	string	string	String (UTF-8)
bytes	May contain any arbitrary sequence of bytes.	string	ByteString	str	[]byte	ByteString	string	String (ASCII-8BIT)

Feedback

Was this page helpful?

Yes No

Glad to hear it! Please tell us how we can improve.

Sorry to hear that. Please tell us how we can improve.