JapaneseContextAnalysis Class Reference

#include <JpCntx.h>

Inheritance diagram for JapaneseContextAnalysis:

Public Member Functions
	JapaneseContextAnalysis ()
virtual	~JapaneseContextAnalysis ()
void	HandleData (const char *aBuf, PRUint32 aLen)
void	HandleOneChar (const char *aStr, PRUint32 aCharLen)
float	GetConfidence ()
void	Reset (void)
void	SetOpion ()
PRBool	GotEnoughData ()
Protected Member Functions
virtual PRInt32	GetOrder (const char str, PRUint32 charLen)=0
virtual PRInt32	GetOrder (const char *str)=0
Protected Attributes
PRUint32	mRelSample [NUM_OF_CATEGORY]
PRUint32	mTotalRel
PRInt32	mLastCharOrder
PRUint32	mNeedToSkipCharNum
PRBool	mDone

Detailed Description

Definition at line 51 of file JpCntx.h.

Constructor & Destructor Documentation

JapaneseContextAnalysis::JapaneseContextAnalysis ( ) [inline]

Definition at line 54 of file JpCntx.h.

References Reset().

00054 {Reset();};

Here is the call graph for this function:

virtual JapaneseContextAnalysis::~JapaneseContextAnalysis ( ) [inline, virtual]

Definition at line 55 of file JpCntx.h.

00055 {}

Member Function Documentation

float JapaneseContextAnalysis::GetConfidence ( )

Definition at line 184 of file JpCntx.cpp.

References DONT_KNOW, MINIMUM_DATA_THRESHOLD, mRelSample, and mTotalRel.

Referenced by nsSJISProber::GetConfidence(), and nsEUCJPProber::GetConfidence().

00185 {
00186   //This is just one way to calculate confidence. It works well for me.
00187   if (mTotalRel > MINIMUM_DATA_THRESHOLD)
00188     return ((float)(mTotalRel - mRelSample[0]))/mTotalRel;
00189   else 
00190     return (float)DONT_KNOW;
00191 }

Here is the caller graph for this function:

virtual PRInt32 JapaneseContextAnalysis::GetOrder ( const char * str ) [protected, pure virtual]

Implemented in SJISContextAnalysis, and EUCJPContextAnalysis.

virtual PRInt32 JapaneseContextAnalysis::GetOrder	(	const char *	str,
		PRUint32 *	charLen
	)			`[protected, pure virtual]`

Implemented in SJISContextAnalysis, and EUCJPContextAnalysis.

Referenced by HandleData(), and HandleOneChar().

Here is the caller graph for this function:

PRBool JapaneseContextAnalysis::GotEnoughData ( ) [inline]

Definition at line 81 of file JpCntx.h.

References ENOUGH_REL_THRESHOLD, and mTotalRel.

Referenced by nsSJISProber::HandleData(), and nsEUCJPProber::HandleData().

00081 {return mTotalRel > ENOUGH_REL_THRESHOLD;};

Here is the caller graph for this function:

void JapaneseContextAnalysis::HandleData	(	const char *	aBuf,
		PRUint32	aLen
	)

Definition at line 131 of file JpCntx.cpp.

References GetOrder(), jp2CharContext, MAX_REL_THRESHOLD, mDone, mLastCharOrder, mNeedToSkipCharNum, mRelSample, mTotalRel, and PR_TRUE.

00132 {
00133   PRUint32 charLen;
00134   PRInt32 order;
00135   PRUint32 i;
00136   
00137   if (mDone)
00138     return;
00139 
00140   //The buffer we got is byte oriented, and a character may span in more than one
00141   //buffers. In case the last one or two byte in last buffer is not complete, we 
00142   //record how many byte needed to complete that character and skip these bytes here.
00143   //We can choose to record those bytes as well and analyse the character once it 
00144   //is complete, but since a character will not make much difference, by simply skipping
00145   //this character will simply our logic and improve performance.
00146   for (i = mNeedToSkipCharNum; i < aLen; )
00147   {
00148     order = GetOrder(aBuf+i, &charLen);
00149     i+= charLen;
00150     if (i > aLen){
00151       mNeedToSkipCharNum = i - aLen;
00152       mLastCharOrder = -1;
00153     }
00154     else 
00155     {
00156       if (order != -1 && mLastCharOrder != -1)
00157       {
00158         mTotalRel ++;
00159         if (mTotalRel > MAX_REL_THRESHOLD)
00160         {
00161           mDone = PR_TRUE;
00162           break;
00163         }
00164         mRelSample[jp2CharContext[mLastCharOrder][order]]++;
00165       }
00166       mLastCharOrder = order;
00167     }
00168   }
00169   
00170   return;
00171 }

Here is the call graph for this function:

void JapaneseContextAnalysis::HandleOneChar	(	const char *	aStr,
		PRUint32	aCharLen
	)			`[inline]`

Definition at line 59 of file JpCntx.h.

References GetOrder(), jp2CharContext, MAX_REL_THRESHOLD, mDone, mLastCharOrder, mRelSample, mTotalRel, and PR_TRUE.

Referenced by nsSJISProber::HandleData(), and nsEUCJPProber::HandleData().

00060   {
00061     PRInt32 order;
00062 
00063     //if we received enough data, stop here   
00064     if (mTotalRel > MAX_REL_THRESHOLD)   mDone = PR_TRUE;
00065     if (mDone)       return;
00066      
00067     //Only 2-bytes characters are of our interest
00068     order = (aCharLen == 2) ? GetOrder(aStr) : -1;
00069     if (order != -1 && mLastCharOrder != -1)
00070     {
00071       mTotalRel++;
00072       //count this sequence to its category counter
00073       mRelSample[jp2CharContext[mLastCharOrder][order]]++;
00074     }
00075     mLastCharOrder = order;
00076   };

Here is the call graph for this function:

Here is the caller graph for this function:

void JapaneseContextAnalysis::Reset ( void )

Definition at line 173 of file JpCntx.cpp.

References mDone, mLastCharOrder, mNeedToSkipCharNum, mRelSample, mTotalRel, NUM_OF_CATEGORY, and PR_FALSE.

Referenced by JapaneseContextAnalysis(), nsSJISProber::Reset(), and nsEUCJPProber::Reset().

00174 {
00175   mTotalRel = 0;
00176   for (PRUint32 i = 0; i < NUM_OF_CATEGORY; i++)
00177     mRelSample[i] = 0;
00178   mNeedToSkipCharNum = 0;
00179   mLastCharOrder = -1;
00180   mDone = PR_FALSE;
00181 }