root/trac/hacks/marketplugin/0.9/TODO

x fix hdf paths 
x cmd 'wiki.SandBox.3' shows order book
x trader acct setup
x specialist env setup
x cmd 'wiki.SandBox' to show basket -- use wiki IRequestHandler interface
x basket match split into component legs
x wiki specialist code needs to be marketmaker so traders can buy/sell baskets
    ...do this by overriding quote(), doing transactions to ensure that
        specialist has at least 500 contracts at bid and ask at all times
x test buying/selling baskets
x WIP
x rename 'cash' to 'available'
x rename ISpecialist to ICommandInterpreter
x IWikiModel api patch
x reporting interpreter
x portfolio interpreter
x clean up Mock in test_wiki, using IWikiModel api
x splits -- use wiki change listener
x housekeeping code in post_cmd()
x make it possible to place orders via html form
x basket sliders
x cmd_batch
x finish Response class 
x rework db classes to inherit from dict
x move all hdf population and template code into web_ui, just return
x     Response objects from cmd_*
x fix __iter__ everywhere so it only returns keys
x get rid of intf parm
x ensure transactions can span cursor create/delete
x only get_db_cnx in ui
x remove all commits; only commit or rollback in ui
x use top-level market.cs to select templates based on namespace
  x each template refers to verb to select content
x interleave wiki.* into market.basket.*
x add grants to housekeeping code
x base seed on number of issued baskets, then use e.g.
  WikiSplitter.wiki_page_*() to ensure points issued = traders * pages
x rename report to portfolio
x fix URLs
x (green, yellow, red) message box
x don't manipulate market.info etc. directly -- write/use Response.info()
x fix order flow
x rename wiki.cs to market/wiki-override.cs; use IRequestPostProcessor
x what to do about wiki.cs? -- override it; we don't disable wiki
  x component, but instead intervene via IWikiWorkflowController and
  x overridden wiki.cs
x test/fix order cancellation via web
x test grants
x logging
x "publish" button on bottom of wiki page, using IRequestPostProcessor
x write shellauth plugin
x publish: new high needs to stay locked, not be normalized
x NonPublic
x upgrade t7a trac sites
x install @ t7a

x wiki.cs ctxtnav: 
  x move ctxtnav into its own template, include in wiki_override.cs
  x page, book, portfolio links
  x don't disable links when on target page; handy for reload
  x current balance
  x fix portfolio page
x fix page creation
x write mq.Bus
x write test_mq
x create Controller; it creates bus, db, and ctx; ctl.run_cmd() returns
  entire res tree
x have res._set publish to bus
x make url RESTful
x write tests.util.run_ui
x rework web_ui to be a messaging endpoint
x make get_default_version also look at bid and ask
x use tracd to test on laptop


- rename mq to pyqle
- create message table
- move bus into standalone daemon
- move order matching etc into standalone daemon(s)

- fix publish button


- fix whatever global is breaking re-used code, then set
  MaxRequestsPerChild back to 0


- primary goal; break entire app up into lots of little modules which
  communicate via message bus; make entire app a messaging endpoint by
  providing a messaging api to *_ui; keep complexity out of core, move
  it to edges ("move conditionals to edges")



- deprecate res_*
- deprecate Command.res()

- follow buy order through to see how well the following would work...

- order db, matching, etc. needs to be moved to backend market server;
  just use fe servers for content.
- fqdn reverse and prepend to all non-currency symbols going into
  market server, strip on the way out
- require registry of currency with market server, rights to use
- 'Market' link takes you to market server
- switch between local and remote market via config url
- comms:
    - just use xmlrpc to ship Msg to market server 
    - or hold a TCP connection open so messages can flow
      bidirectionally
- webkdc to auth user
- test backend as remote daemon; use setUp and tearDown to stop and
  start
- use Shibboleth to authorize frontend server someday

- migrate ILedger code to IMessageBusSubscriber, replace ILedger with
  bus client code

- write REST_ui, where last component of URL == path on message bus
- Use bus reader implementing REST client to talk to remote ledger
      
- deprecate run_cmd

- find and deprecate all references to res.* (about 11): 
    svngrep 'res[\.\[]' | grep -v test_wiki | egrep '[(=].*res[\.\[]'  

- shut off res storage in res._set
- NOT have res subscribe to bus and accumulate data
- find and deprecate all assignments to res.* (about 100): 
    svngrep 'res[\.\[]' | egrep 'res[\.\[].*='
- add a deprecation message to res.__getitem__
- delete most of res

- ...either way, always send complete data structures to res.*
  groups; avoid having to deal with indexes or dict keys in msg attr;
  this is already the case; Response.append is only used by Response

    
- use bus to propagate messages
- rewrite res.info, warning, error to send to bus rather than store;
  need an 'append' msg attr for this

- re-enable resdump form arg

- deprecate response_cmd; have cmd_* handle next view



- make no changes without first seeing/writing a failing test case 


UI

- basket page
  x add asterisk to show highest price
  x column to take you to page, column to take you to order book
  x add bargraph
  - description of page as subtitle 
  - help page

- content page
  x clean up page_ctl; wrap in table etc.

- comments on orders

- wip publishing

- popup js graphs e.g.
  http://wwwsmartmoney.com/etffocus/index.cfm?story=20060329

MODEL/CODE

- write market data reporting, use as default market home page 
  - "Things to do"
    - descriptive paragraph, or ref wiki help page
    - "Pages to review" -- pages that have not been traded recently
        - contracts show_volume bottom 5
    - "Pages to edit" -- pages with wip as highest price
        - wiki show_wip top 5
    - "Deletion candidates" -- contracts with lowest price
        - contracts show_value bottom 5
  - "Highest Rated" -- contracts with highest price
    - contracts show_value top 5
  - "Most Active" -- contracts with highest recent volume of orders
    - contracts show_volume top 5
  - "Top Traders" -- available - granted
    x traders show_worth top 5
    - traders show_trades top 5

- wiki link formatting for e.g. market:wiki.FooPage.3

- add bus reader methods to Order class, and remove all direct
  references to Order -- there are only about 22, most in
  specialist.py

- move Context to util, use in web_ui
- deprecate ctx.symbol, verb, args, func, and all other per-request
  attributes in favor of bus and port attributes, doublecheck that
  this makes redirects easier

- model classes e.g. Symbol are instrument-specific, so go ahead and define
  them in specialist children, even though that binds model and controller
  together too tightly; use message bus to get to them

- deprecate all direct calls to db from interpreter in favor of calls
  via message bus

- move matching and other business logic to models
 

THIS WEEK

- add testWebUI test

- populate ledger ref info 

- portfolio too slow -- disabled history for now

LATER

- get db.commit() out of web_ui?  dispatch_request does db.close()...

- prevent duplication of data and effort: convert to using a make-like
  algorithm for populating res; each cmd_* calls a series of res_*
  methods, each method is named after its res subtree and vice
  versa, each checks to see if the data is already populated before
  creating it again; res is written by res_* only; any method can
  call any res_* method and use the results

- XXX or just hang all that and turn every symbol into an object which
  calculates and caches its own quote, position, order, and other data

- get rid of all direct ctx.res references

- move OrderQ to order.py

- get rid of show_*, use subcount()

- handle database lock/retry in ui
- move all mentions of req out of command interpreters; use authname in ctx

- migrate to fixed-place decimal from float -- change db data type to
  (13,3) precision
  - alternative is to just use SQL round(X,Y) during reconciliation 

- rescale tests to basket_price = 100

patch: move handle_ta until after change_listener calls
patch: pass db to change_listener
use db in change_listener, kill that get_db_cnx

scripting language, with optional rollback on error

currency exchange 
    fx.USD b 100 5.5 
    fx.USD s 15 6.2 

withdrawals -- cash
    from 'payable' account
    portfolio withdraw 67 USD
deposits -- credit card, cash
    into 'receivable' account
    support POST...
    portfolio deposit 67 USD
    > 1234 5678 9012 3456
    ...resulting ctx object includes instructions for ui to get stdin or POST
balance transfers out of own avail acct
    portfolio pay foo 23 TMP memo jslkajfd slkjfd klsa flak

write Auditor.cmd_reconcile

implement cli interface
write crashme 

migrate interfaces to api.py, clean up imports

indexes
permissions
timeline provider
autodelete page versions below a configurable threshold price and last
    trade age

allowance for activity 
    number of pages whose default viewable were created by trader

handle e.g. wiki page content updates via cli
    wiki.FooPage content
    > jfdsa lkjfdsa lksjf lsakjf lsakjf 
    > ...


trade arbitrary products via 'market' specialist
    market.blah buy 67 .23 TMP 
    market.blah buy 67 .23 TMP from foo
    market.blah sell 67 .23 TMP to foo


TENTATIVE

- model requirements/implementation
  - cache results of db lookups and other expensive operations
  - build a res tree that is same namespace as url path
    - market/wiki.FooPage.3 -> market.wiki.FooPage.3.*
    - market/portfolio -> market.portfolio.balance.*
    - cmd_* are responsible for maintaining this tree
  - works like HSM
    - "inodes" are IMarketModel implementors; they register path
      support via inode = model.open_path(path)
    - "data blocks" are results of inode(verb, args)
    - 'wiki.FooPage.3 buy 3 .23' causes:
        inode = model.open('wiki.FooPage.3')
        inode.buy(.23)
    - refactor ICommandInterpreter to use IMarketModel
  - any model must be equally accessible from any controller
    - pretty much dictates use of IMarketModel
    - model.factory(ctx, name, id) returns the model obj


  - move all business logic out of controllers
    - can be in same module, but not in same class
    - need to move most config.get() into models; probably need a
      Config model
  - clean access syntax (avoid explicit search loops)
    - wrapper in Context
    - ctx.Symbol('wiki.FooPage') calls model.factory(ctx.res,
      'Symbol', 'wiki.FooPage') on each model, stores the resulting
      obj when it gets one back
  - fast access (avoid search loops altogether)
    - ctx stores model obj for re-use, in ctx.model[name][args]
  - works for both sync and async apps
    - controller passes View obj to model constructors, e.g.
      model.factory(ctx, ctx.res, 'Symbol', 'wiki.FooPage') creates a model
      which will write to ctx.res.Symbol.wiki.FooPage.*
    - frontend models can create and pass a different view obj to
      backend models
      - Symbol creates view = View() and passes that to Order, Order
        writes to view.Order[id].*
    - model sets viewobj.model_class.* whenever data is fetched/changed
      - DbObject.insert and DbObject.update set
        viewobj.[table_name][id].*


- root model classes are Symbol, Order, Quote, Account, Entity, Deal

- data path is controller -> model -> viewobj
- e.g. cmd_show -> Symbol (calls db) -> ctx.res.Symbol



- clean up basket when hitting next symbol-related bug > 1 hour:  Just
  clean up model and this will take care of itself.  XXX not this:
  Start by grepping for the variable 'basket', and removing it by
  turning all uses of 'symbol' from a string into Symbol class; a
  Symbol can contain other contracts, recursiveley, with each Symbol
  also having a 'basket' attr; wiki.Symbol inherits from
  command.Symbol; probably move all non-cmd_* methods from Specialist
  to Symbol or other models