Core Builtin Functions¶
BOMLOptimizer.meta_learner:
- Aliases:
- boml.boml_optimizer.BOMLOptimizer.meta_learner()
boml.boml_optimizer.BOMLOptimizer.meta_learner( _input, dataset, meta_model='V1', name='Hyper_Net', use_t=False, use_warp=False, **model_args )
This method must be called once at first to build meta modules and initialize meta parameters and neural networks.
Args:
- _input: orginal input for neural network construction;
- dataset: which dataset to use for training and testing. It should be initialized before being passed into the function
- meta_model: model chosen for neural network construction, V1 for C4L with fully connected layer,`V2` for Residual blocks with fully connected layer.
- name: name for Meta model modules used for BOMLNet initialization
- use_t: whether to use T layer for C4L neural networks
- use_warp: whether to use Warp layer for C4L neural networks
- model_args: optional arguments to set specific parameters of neural networks.
BOMLOptimizer.base_learner:
- Aliases:
- boml.boml_optimizer.BOMLOptimizer.base_learner()
boml.boml_optimizer.BOMLOptimizer.base_learner( _input, meta_learner, name='Task_Net', weights_initializer=tf.zeros_initializer )
This method has to be called for every experiment and takes responsibility for defining task-specific modules and inner optimizer.
Args:
- _input: orginal input for neural network construction of task-specific module;
- meta_learner: returned value of meta_learner function, which is a instance of BOMLNet or its child classes
- name: name for Base model modules used for BOMLNet initialization
- weights_initializer: initializer function for task_specific network, called by ‘MetaRepr’ method
Returns: task-specific model part
BOMLOptimizer.ll_problem:
- Aliases:
- boml.boml_optimizer.BOMLOptimizer.ll_problem()
boml.boml_optimizer.BOMLOptimizer.ll_problem( inner_objective, learning_rate, T, inner_objective_optimizer='SGD', outer_objective=None, learn_lr=False, alpha_init=0.0, s=1.0, t=1.0, learn_alpha=False, learn_st=False, learn_alpha_itr=False, var_list=None, init_dynamics_dict=None, first_order=False, loss_func=utils.cross_entropy, momentum=0.5, beta1=0.0, beta2=0.999, regularization=None, experiment=None, scalor=0.0, **inner_kargs )
After construction of neural networks, solutions to lower level problems should be regulated in ll_problem.
- Args:
- inner_objective: loss function for the inner optimization problem
- learning_rate: step size for inner loop optimization
- T: numbers of steps for inner gradient descent optimization
- inner_objective_optimizer: Optimizer type for the outer parameters, should be in list [SGD,`Momentum`,`Adam`]
- outer_objective: loss function for the outer optimization problem, which need to be claimed in BDA agorithm
- alpha_init: initial value of ratio of inner objective to outer objective in BDA algorithm
- s,t: coefficients of aggregation of inner and outer objectives in BDA algorithm, default to be 1.0
- learn_alpha: specify parameter for BDA algorithm to decide whether to initialize alpha as a hyper parameter
- learn_alpha_itr: parameter for BDA algorithm to specify whether to initialize alpha as a vector, of which every dimension’s value is step-wise scale factor fot the optimization process
- learn_st: specify parameter for BDA algorithm to decide whether to initialize s and t as hyper parameters
- first_order: specific parameter to define whether to use implement first order MAML, default to be FALSE
- loss_func: specifying which type of loss function is used for the maml-based method, which should be consistent with the form to compute the inner objective
- momentum: specific parameter for Optimizer.BOMLOptMomentum to set initial value of momentum
- regularization: whether to add regularization terms in the inner objective
- experiment: instance of Experiment to use in the Lower Level Problem, especifially needed in the MetaRper type of method.
- var_list: optional list of variables (of the inner optimization problem)
- inner_kargs: optional arguments to pass to boml.boml_optimizer.BOMLOptimizer.compute_gradients
- Returns: task-specific model part
BOMLOptimizer.ul_problem
Aliases:
boml.boml_optimizer.BOMLOptimizer.ul_problem()
boml.boml_optimizer.BOMLOptimizer.ul_Problem( outer_objective, meta_learning_rate, inner_grad, meta_param=None, outer_objective_optimizer='Adam', epsilon=1.0, momentum=0.5, global_step=None )
This method define upper level problems and choose optimizer to optimize meta parameters, which should be called afer ll_problem.
Args:
- outer_objective: scalar tensor for the outer objective
- meta_learning_rate: step size for outer loop optimization
- inner_grad: Returned value of boml.BOMLOptimizer.LLProblem()
- meta_param: optional list of outer parameters and model parameters
- outer_objective_optimizer: Optimizer type for the outer parameters, should be in list [SGD,`Momentum`,`Adam`]
- epsilon: Float, cofffecients to be used in DARTS algorithm
- momentum: specific parameters to be used to initialize Momentum algorithm
Returns:meta_param list, used for debugging
aggregate_all:
Aliases:
- boml.boml_optimizer.BOMLOptimizer.aggregate_all()
- ::
- boml.boml_optimizer.BOMLOptimizer.aggregate_all(
aggregation_fn=None, gradient_clip=None )
- Args:
- aggregation_fn:Optional operation to aggregate multiple outer_gradients (for the same meta parameter),by (default: reduce_mean)
- gradient_clip: optional operation to clip the aggregated outer gradients
Returns: None
Finally, aggregate_all has to be called to aggregate gradient of different tasks, and define operations to apply outer gradients and update meta parametes.
run:
- Aliases:
- boml.boml_optimizer.BOMLOptimizer.run()
boml.boml_optimizer.BOMLOptimizer.run( inner_objective_feed_dicts=None, outer_objective_feed_dicts=None, session=None, _skip_hyper_ts=False, _only_hyper_ts=False, callback=None )
- Args:
- inner_objective_feed_dicts: an optional feed dictionary for the inner problem. Can be a function of step, which accounts for, e.g. stochastic gradient descent.
- outer_objective_feed_dicts: an optional feed dictionary for the outer optimization problem (passed to the evaluation of outer objective). Can be a function of hyper-iterations steps (i.e. global variable), which may account for, e.g. stochastic evaluation of outer objective.
- session: optional session
- callback: optional callback function of signature (step (int), feed_dictionary, tf.Session) -> None that are called after every forward iteration.
Returns: None