edu.stanford.nlp.classify

Class LogConditionalObjectiveFunction<L,F>

java.lang.Object

edu.stanford.nlp.optimization.AbstractCachingDiffFunction

edu.stanford.nlp.optimization.AbstractStochasticCachingDiffFunction

edu.stanford.nlp.optimization.AbstractStochasticCachingDiffUpdateFunction

edu.stanford.nlp.classify.LogConditionalObjectiveFunction<L,F>

All Implemented Interfaces:

DiffFunction, Function, HasInitial

Direct Known Subclasses:

AdaptedGaussianPriorObjectiveFunction

public class LogConditionalObjectiveFunction<L,F>
extends AbstractStochasticCachingDiffUpdateFunction

Maximizes the conditional likelihood with a given prior.

Author:

Dan Klein, Galen Andrew, Chris Cox (merged w/ SumConditionalObjectiveFunction, 2/16/05), Sarah Spikes (Templatization, allowing an Iterable> to be passed in instead of a GeneralDataset), Angel Chang (support in place SGD - extend AbstractStochasticCachingDiffUpdateFunction)

Nested Class Summary

Nested classes/interfaces inherited from class edu.stanford.nlp.optimization.AbstractStochasticCachingDiffFunction

AbstractStochasticCachingDiffFunction.SamplingMethod

Field Summary

Fields

Modifier and Type	Field and Description
protected int[][]	data
protected java.lang.Iterable<Datum<L,F>>	dataIterable
protected float[]	dataweights
protected DoubleAD[]	derivativeAD
protected double[]	derivativeNumerator
protected Index<F>	featureIndex
protected Index<L>	labelIndex
protected int[]	labels
protected int	numClasses
protected int	numFeatures
protected LogPrior	prior
protected double[]	priorDerivative
protected DoubleAD[]	probs
protected DoubleAD[]	sums
protected boolean	useIterable
protected boolean	useSummedConditionalLikelihood
protected double[][]	values
protected DoubleAD[]	xAD

Fields inherited from class edu.stanford.nlp.optimization.AbstractStochasticCachingDiffUpdateFunction

skipValCalc

Fields inherited from class edu.stanford.nlp.optimization.AbstractStochasticCachingDiffFunction

allIndices, curElement, finiteDifferenceStepSize, gradPerturbed, hasNewVals, HdotV, lastBatch, lastBatchSize, lastElement, lastVBatch, lastXBatch, method, randGenerator, recalculatePrevBatch, returnPreviousValues, sampleMethod, scaleUp, thisBatch, xPerturbed

Fields inherited from class edu.stanford.nlp.optimization.AbstractCachingDiffFunction

derivative, generator, value

Constructor Summary

Constructors

Constructor and Description
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset)
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset, float[] dataWeights, LogPrior prior)
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset, LogPrior prior)
LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset, LogPrior prior, boolean useSumCondObjFun)
LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, double[][] values, int[] labels, int intPrior, double sigma, double epsilon)
LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels)
LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels, boolean useSumCondObjFun)
LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels, float[] dataweights)
LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels, float[] dataweights, LogPrior prior)
LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels, int intPrior, double sigma, double epsilon)
LogConditionalObjectiveFunction(int numFeatures, int numClasses, int[][] data, int[] labels, LogPrior prior)
LogConditionalObjectiveFunction(java.lang.Iterable<Datum<L,F>> dataIterable, LogPrior logPrior, Index<F> featureIndex, Index<L> labelIndex)

Method Summary

Methods

Modifier and Type	Method and Description
protected void	calculate(double[] x) Calculate the conditional likelihood.
void	calculateStochastic(double[] x, double[] v, int[] batch) calculateStochastic needs to calculate a stochastic approximation to the derivative and value of of a function for a given batch of the data.
protected void	calculateStochasticAlgorithmicDifferentiation(double[] x, double[] v, int[] batch)
void	calculateStochasticFiniteDifference(double[] x, double[] v, double h, int[] batch)
void	calculateStochasticGradient(double[] x, int[] batch) Performs stochastic gradient calculation based on samples indexed by batch and do not apply regularization.
void	calculateStochasticGradientLocal(double[] x, int[] batch)
double	calculateStochasticUpdate(double[] x, double xscale, int[] batch, double gain) Performs stochastic update of weights x (scaled by xScale) based on samples indexed by batch.
int	dataDimension() Data dimension must return the size of the data used by the function.
int	domainDimension() Returns the number of dimensions in the function's domain
protected int	indexOf(int f, int c)
protected void	rvfcalculate(double[] x) Calculate conditional likelihood for datasets with real-valued features.
void	setPrior(LogPrior prior)
void	setUseSumCondObjFun(boolean value)
double[][]	to2D(double[] x)
double	valueAt(double[] x, double xscale, int[] batch) Computes value of function for specified value of x (scaled by xScale) only over samples indexed by batch.

Methods inherited from class edu.stanford.nlp.optimization.AbstractStochasticCachingDiffUpdateFunction

calculateStochasticGradient, calculateStochasticUpdate, getSample, valueAt

Methods inherited from class edu.stanford.nlp.optimization.AbstractStochasticCachingDiffFunction

clearCache, decrementBatch, derivativeAt, derivativeAt, getBatch, HdotVAt, HdotVAt, HdotVAt, incrementBatch, incrementRandom, initial, lastDerivative, lastValue, scaleUp, valueAt, valueAt

Methods inherited from class edu.stanford.nlp.optimization.AbstractCachingDiffFunction

copy, derivativeAt, ensure, getDerivative, gradientCheck, gradientCheck, randomInitial, valueAt

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

prior

protected LogPrior prior

numFeatures

protected int numFeatures

numClasses

protected int numClasses

data

protected int[][] data

dataIterable

protected java.lang.Iterable<Datum<L,F>> dataIterable

values

protected double[][] values

labels

protected int[] labels

dataweights

protected float[] dataweights

derivativeNumerator

protected double[] derivativeNumerator

xAD

protected DoubleAD[] xAD

priorDerivative

protected double[] priorDerivative

derivativeAD

protected DoubleAD[] derivativeAD

sums

protected DoubleAD[] sums

probs

protected DoubleAD[] probs

labelIndex

protected Index<L> labelIndex

featureIndex

protected Index<F> featureIndex

useIterable

protected boolean useIterable

useSummedConditionalLikelihood

protected boolean useSummedConditionalLikelihood

Constructor Detail

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset)

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
                               LogPrior prior)

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
                               float[] dataWeights,
                               LogPrior prior)

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(GeneralDataset<L,F> dataset,
                               LogPrior prior,
                               boolean useSumCondObjFun)

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(java.lang.Iterable<Datum<L,F>> dataIterable,
                               LogPrior logPrior,
                               Index<F> featureIndex,
                               Index<L> labelIndex)

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(int numFeatures,
                               int numClasses,
                               int[][] data,
                               int[] labels,
                               boolean useSumCondObjFun)

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(int numFeatures,
                               int numClasses,
                               int[][] data,
                               int[] labels)

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(int numFeatures,
                               int numClasses,
                               int[][] data,
                               int[] labels,
                               LogPrior prior)

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(int numFeatures,
                               int numClasses,
                               int[][] data,
                               int[] labels,
                               float[] dataweights)

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(int numFeatures,
                               int numClasses,
                               int[][] data,
                               int[] labels,
                               float[] dataweights,
                               LogPrior prior)

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(int numFeatures,
                               int numClasses,
                               int[][] data,
                               int[] labels,
                               int intPrior,
                               double sigma,
                               double epsilon)

LogConditionalObjectiveFunction

public LogConditionalObjectiveFunction(int numFeatures,
                               int numClasses,
                               int[][] data,
                               double[][] values,
                               int[] labels,
                               int intPrior,
                               double sigma,
                               double epsilon)

Method Detail

setPrior

public void setPrior(LogPrior prior)

domainDimension

public int domainDimension()

Description copied from interface: Function

Returns the number of dimensions in the function's domain

Returns:

the number of domain dimensions

dataDimension

public int dataDimension()

Description copied from class: AbstractStochasticCachingDiffFunction

Data dimension must return the size of the data used by the function.

Specified by:

dataDimension in class AbstractStochasticCachingDiffFunction

indexOf

protected int indexOf(int f,
          int c)

to2D

public double[][] to2D(double[] x)

calculate

protected void calculate(double[] x)

Calculate the conditional likelihood. If useSummedConditionalLikelihood is false (the default), this calculates standard(product) CL, otherwise this calculates summed CL. What's the difference? See Klein and Manning's 2002 EMNLP paper.

Specified by:

calculate in class AbstractCachingDiffFunction

Parameters:

x - The point at which to calculate the function

calculateStochastic

public void calculateStochastic(double[] x,
                       double[] v,
                       int[] batch)

Description copied from class: AbstractStochasticCachingDiffFunction

calculateStochastic needs to calculate a stochastic approximation to the derivative and value of of a function for a given batch of the data. The approximation to the derivative must be stored in the array derivative , the approximation to the value in value and the approximation to the Hessian vector product H.v in the array HdotV . Note that the hessian vector product is used primarily with the Stochastic Meta Descent optimization routine SMDMinimizer . Important: The stochastic approximation must be such that the sum of all stochastic calculations over each of the batches in the data must equal the full calculation. i.e. for a data set of size 100 the sum of the gradients for batches 1-10 , 11-20 , 21-30 .... 91-100 must be the same as the gradient for the full calculation (at the very least in expectation). Be sure to take into account the priors.

Specified by:

calculateStochastic in class AbstractStochasticCachingDiffFunction

Parameters:

x - - value to evaluate at

v - - the vector for the Hessian vector product H.v

batch - - an array containing the indices of the data to use in the calculation, this array is being calculated internal to the abstract, and only needs to be handled not generated by the implementation.

calculateStochasticFiniteDifference

public void calculateStochasticFiniteDifference(double[] x,
                                       double[] v,
                                       double h,
                                       int[] batch)

calculateStochasticGradientLocal

public void calculateStochasticGradientLocal(double[] x,
                                    int[] batch)

valueAt

public double valueAt(double[] x,
             double xscale,
             int[] batch)

Description copied from class: AbstractStochasticCachingDiffUpdateFunction

Computes value of function for specified value of x (scaled by xScale) only over samples indexed by batch.

Specified by:

valueAt in class AbstractStochasticCachingDiffUpdateFunction

Parameters:

x - unscaled weights

xscale - how much to scale x by when performing calculations

batch - indices of which samples to compute function over

Returns:

value of function at specified x (scaled by xScale) for samples

calculateStochasticUpdate

public double calculateStochasticUpdate(double[] x,
                               double xscale,
                               int[] batch,
                               double gain)

Description copied from class: AbstractStochasticCachingDiffUpdateFunction

Performs stochastic update of weights x (scaled by xScale) based on samples indexed by batch.

Specified by:

calculateStochasticUpdate in class AbstractStochasticCachingDiffUpdateFunction

Parameters:

x - unscaled weights

xscale - how much to scale x by when performing calculations

batch - indices of which samples to compute function over

gain - how much to scale adjustments to x

Returns:

value of function at specified x (scaled by xScale) for samples

calculateStochasticGradient

public void calculateStochasticGradient(double[] x,
                               int[] batch)

Description copied from class: AbstractStochasticCachingDiffUpdateFunction

Performs stochastic gradient calculation based on samples indexed by batch and do not apply regularization. does not update the parameter values

Specified by:

calculateStochasticGradient in class AbstractStochasticCachingDiffUpdateFunction

Parameters:

x - unscaled weights

batch - indices of which samples to compute function over

calculateStochasticAlgorithmicDifferentiation

protected void calculateStochasticAlgorithmicDifferentiation(double[] x,
                                                 double[] v,
                                                 int[] batch)

rvfcalculate

protected void rvfcalculate(double[] x)

Calculate conditional likelihood for datasets with real-valued features. Currently this can calculate CL only (no support for SCL). TODO: sum-conditional obj. fun. with RVFs.

setUseSumCondObjFun

public void setUseSumCondObjFun(boolean value)