Detailed Description

Class SVRLight, performs support vector regression using SVMLight.

The SVR solution can be expressed as

$f({\bf x})=\sum_{i=1}^{N} \alpha_i k({\bf x}, {\bf x_i})+b$

where $\alpha$ and $b$ are determined in training, i.e. using a pre-specified kernel, a given tube-epsilon for the epsilon insensitive loss, the follwoing quadratic problem is minimized (using the chunking decomposition technique)

$\begin{eqnarray*} \max_{{\bf \alpha},{\bf \alpha}^*} &-\frac{1}{2}\sum_{i,j=1}^N(\alpha_i-\alpha_i^*)(\alpha_j-\alpha_j^*){\bf x}_i^T {\bf x}_j -\sum_{i=1}^N(\alpha_i+\alpha_i^*)\epsilon - \sum_{i=1}^N(\alpha_i-\alpha_i^*)y_i\\ \mbox{wrt}:& {\bf \alpha},{\bf \alpha}^*\in{\bf R}^N\\ \mbox{s.t.}:& 0\leq \alpha_i,\alpha_i^*\leq C,\, \forall i=1\dots N\\ &\sum_{i=1}^N(\alpha_i-\alpha_i^*)y_i=0 \end{eqnarray*}$

Note that the SV regression problem is reduced to the standard SV classification problem by introducing artificial labels $-y_i$ which leads to the epsilon insensitive loss constraints *

$\begin{eqnarray*} {\bf w}^T{\bf x}_i+b-c_i-\xi_i\leq 0,&\, \forall i=1\dots N\\ -{\bf w}^T{\bf x}_i-b-c_i^*-\xi_i^*\leq 0,&\, \forall i=1\dots N \end{eqnarray*}$

with $c_i=y_i+ \epsilon$ and $c_i^*=-y_i+ \epsilon$

This implementation supports multiple kernel learning, i.e. if a CCombinedKernel is used the weights $\beta$ in $k_{combined}({\bf x}, {\bf x'}) = \sum_{m=0}^M \beta_m k_m({\bf x}, {\bf x'})$ can be determined in training (cf. Large Scale Multiple Kernel Learning Sonnenburg, Raetsch, Schaefer, Schoelkopf 2006).

linadd optimizations were implemented for kernels that support it (most string kernels and the linear kernel), which will result in significant speedups.

Definition at line 62 of file SVR_light.h.

Inheritance diagram for CSVRLight:

[legend]

List of all members.

Public Member Functions
	CSVRLight ()
	CSVRLight (float64_t C, float64_t epsilon, CKernel k, CLabels lab)
virtual	~CSVRLight ()
virtual bool	train (CFeatures *data=NULL)
virtual EClassifierType	get_classifier_type ()
void	svr_learn ()
virtual float64_t	compute_objective_function (float64_t a, float64_t lin, float64_t c, float64_t eps, int32_t *label, int32_t totdoc)
virtual void	update_linear_component (int32_t docs, int32_t label, int32_t active2dnum, float64_t a, float64_t a_old, int32_t working2dnum, int32_t totdoc, float64_t lin, float64_t aicache, float64_t *c)
virtual void	update_linear_component_mkl (int32_t docs, int32_t label, int32_t active2dnum, float64_t a, float64_t a_old, int32_t working2dnum, int32_t totdoc, float64_t lin, float64_t aicache, float64_t *c)
virtual void	update_linear_component_mkl_linadd (int32_t docs, int32_t label, int32_t active2dnum, float64_t a, float64_t a_old, int32_t working2dnum, int32_t totdoc, float64_t lin, float64_t aicache, float64_t *c)
void	call_mkl_callback (float64_t a, int32_t label, float64_t lin, float64_t c, int32_t totdoc)
virtual void	reactivate_inactive_examples (int32_t label, float64_t a, SHRINK_STATE shrink_state, float64_t lin, float64_t c, int32_t totdoc, int32_t iteration, int32_t inconsistent, int32_t docs, float64_t aicache, float64_t *maxdiff)
virtual const char *	get_name () const
Protected Member Functions
int32_t	regression_fix_index (int32_t i)
virtual float64_t	compute_kernel (int32_t i, int32_t j)
Static Protected Member Functions
static void *	update_linear_component_linadd_helper (void *params)
static int32_t	regression_fix_index2 (int32_t i, int32_t num_vectors)
Protected Attributes
int32_t	num_vectors

Constructor & Destructor Documentation

CSVRLight ( )

default constructor

Definition at line 57 of file SVR_light.cpp.

CSVRLight	(	float64_t	C,
		float64_t	epsilon,
		CKernel *	k,
		CLabels *	lab
	)

constructor

Parameters:

C	constant C
epsilon	epsilon
k	kernel
lab	labels

Definition at line 51 of file SVR_light.cpp.

virtual ~CSVRLight ( ) [virtual]

default destructor

Definition at line 78 of file SVR_light.h.

Member Function Documentation

void call_mkl_callback	(	float64_t *	a,
		int32_t *	label,
		float64_t *	lin,
		float64_t *	c,
		int32_t	totdoc
	)

Definition at line 587 of file SVR_light.cpp.

virtual float64_t compute_kernel	(	int32_t	i,
		int32_t	j
	)		`[protected, virtual]`

compute kernel at given index

Parameters:

i	index i
j	index j

Returns:: kernel value at i,j

Reimplemented from CSVMLight.

Definition at line 236 of file SVR_light.h.

float64_t compute_objective_function	(	float64_t *	a,
		float64_t *	lin,
		float64_t *	c,
		float64_t *	eps,
		int32_t *	label,
		int32_t	totdoc
	)		`[virtual]`

compute objective function

Parameters:

a	a
lin	lin
c	c
eps	eps
label	label
totdoc	totdoc

Reimplemented from CSVMLight.

Definition at line 332 of file SVR_light.cpp.

virtual EClassifierType get_classifier_type ( ) [virtual]

get classifier type

Returns:: classifier type SVRLIGHT

Reimplemented from CSVMLight.

Definition at line 94 of file SVR_light.h.

virtual const char* get_name ( ) const [virtual]

Returns:: object name

Reimplemented from CSVMLight.

Definition at line 193 of file SVR_light.h.

void reactivate_inactive_examples	(	int32_t *	label,
		float64_t *	a,
		SHRINK_STATE *	shrink_state,
		float64_t *	lin,
		float64_t *	c,
		int32_t	totdoc,
		int32_t	iteration,
		int32_t *	inconsistent,
		int32_t *	docs,
		float64_t *	aicache,
		float64_t *	maxdiff
	)		`[virtual]`

reactivate inactive examples

Parameters:

label	label
a	a
shrink_state	shrink state
lin	lin
c	c
totdoc	totdoc
iteration	iteration
inconsistent	inconsistent
docs	docs
aicache	ai cache
maxdiff	maxdiff

Reimplemented from CSVMLight.

Definition at line 642 of file SVR_light.cpp.

int32_t regression_fix_index ( int32_t i ) [protected]

regression fix index

Parameters:

i i

Returns:: fix index

Definition at line 207 of file SVR_light.h.

static int32_t regression_fix_index2	(	int32_t	i,
		int32_t	num_vectors
	)		`[static, protected]`

regression fix index2

Parameters:

i	i
num_vectors	number of vectors

Returns:: fix index

Definition at line 221 of file SVR_light.h.

void svr_learn ( )

SVR learn

Definition at line 144 of file SVR_light.cpp.

bool train ( CFeatures * data = NULL ) [virtual]

train regression

Parameters:

data	training data (parameter can be avoided if distance or kernel-based regressors are used and distance/kernels are initialized with train data)

Returns:: whether training was successful

Reimplemented from CSVMLight.

Definition at line 62 of file SVR_light.cpp.

void update_linear_component	(	int32_t *	docs,
		int32_t *	label,
		int32_t *	active2dnum,
		float64_t *	a,
		float64_t *	a_old,
		int32_t *	working2dnum,
		int32_t	totdoc,
		float64_t *	lin,
		float64_t *	aicache,
		float64_t *	c
	)		`[virtual]`

update linear component

Parameters:

docs	docs
label	label
active2dnum	active2dnum
a	a
a_old	a old
working2dnum	working2dnum
totdoc	totdoc
lin	lin
aicache	ai cache
c	c

Reimplemented from CSVMLight.

Definition at line 369 of file SVR_light.cpp.

void * update_linear_component_linadd_helper ( void * params ) [static, protected]

thread helper for update linear component linadd

Parameters:

params

Reimplemented from CSVMLight.

Definition at line 356 of file SVR_light.cpp.

void update_linear_component_mkl	(	int32_t *	docs,
		int32_t *	label,
		int32_t *	active2dnum,
		float64_t *	a,
		float64_t *	a_old,
		int32_t *	working2dnum,
		int32_t	totdoc,
		float64_t *	lin,
		float64_t *	aicache,
		float64_t *	c
	)		`[virtual]`

update linear component MKL

Parameters:

docs	docs
label	label
active2dnum	active2dnum
a	a
a_old	a old
working2dnum	working2dnum
totdoc	totdoc
lin	lin
aicache	ai cache
c	c

Definition at line 467 of file SVR_light.cpp.

void update_linear_component_mkl_linadd	(	int32_t *	docs,
		int32_t *	label,
		int32_t *	active2dnum,
		float64_t *	a,
		float64_t *	a_old,
		int32_t *	working2dnum,
		int32_t	totdoc,
		float64_t *	lin,
		float64_t *	aicache,
		float64_t *	c
	)		`[virtual]`

update linear component MKL linadd

Parameters:

docs	docs
label	label
active2dnum	active2dnum
a	a
a_old	a old
working2dnum	working2dnum
totdoc	totdoc
lin	lin
aicache	ai cache
c	c

Definition at line 540 of file SVR_light.cpp.

Member Data Documentation

int32_t num_vectors [protected]

number of train elements

Definition at line 244 of file SVR_light.h.

The documentation for this class was generated from the following files:

Detailed Description

Public Member Functions

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation