Newsgroups: comp.ai.neural-nets
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news.dfci.harvard.edu!camelot.ccs.neu.edu!nntp.neu.edu!news.eecs.umich.edu!news.mathworks.com!newsgate.duke.edu!interpath!news.interpath.net!news.interpath.net!sas!newshost.unx.sas.com!saswss
From: saswss@hotellng.unx.sas.com (Warren Sarle)
Subject: Re: replace backprop MSE opt. by %correct opt.?
Originator: saswss@hotellng.unx.sas.com
Sender: news@unx.sas.com (Noter of Newsworthy Events)
Message-ID: <E6LBzF.Lo1@unx.sas.com>
Date: Wed, 5 Mar 1997 22:04:27 GMT
X-Nntp-Posting-Host: hotellng.unx.sas.com
References:  <19970304100940.BKAMP@pi0020.kub.nl>
Organization: SAS Institute Inc.
Lines: 22


In article <19970304100940.BKAMP@pi0020.kub.nl>, BKAMP@kub.nl  (Kamp B.) writes:
|> ... 
|> Is it imaginable to alter the backprop algorithm in such a way that
|> the MSE optimization is replaced by a %correct optimization?

%correct is not a differentiable function of the weights, so backprop
will not work. Worse, %correct is piecewise constant, so secants,
subgradients, etc. won't work either without a lot of effort. Neither
have I had much luck with the Nelder-Mead simplex algorithm. Your best
bet is probably simulated annealing; see Lester Ingber's page on
Adaptive Simulated Annealing at http://www.ingber.com/ or other
references in ftp://ftp.sas.com/pub/neural/FAQ2.html on nonlinear
programming.

-- 

Warren S. Sarle       SAS Institute Inc.   The opinions expressed here
saswss@unx.sas.com    SAS Campus Drive     are mine and not necessarily
(919) 677-8000        Cary, NC 27513, USA  those of SAS Institute.
 *** Do not send me unsolicited commercial or political email! ***

