Gallery:模块:图片注释（imagenotes）

注：此项目暂时停止，因为一项新的SoC项目（2008）准备加入此功能！

Imagenotes模块

该模块（仍在开发中，尚未发布）允许你在自己的图片中选择区域用于注释或添加标签。有关该模块开发方面的讨论可以在此找到。

状态

• 仍在开发中，有待进一步完善。完成后就会发布。
• 研究其扩展性。
• 研究非矩形（以及complex 3d）图片区域的选择（使用具有YUI抽象的SVG/VML？）。
• 研究浏览器标签选取的排序及渲染层。（YUI OverlayManager可以管理Overlay的z-order，但无法按照预期改变标签排序。需要对OverlayManager进行修改才能重新排序文档）。
• 客户端方矩形区域选取及编辑（移动，重设尺寸）完成。

各类情况

简单的图片文本标记：

• 用户希望通过标记图片上友人面部区域来对图片进行批注。当某人查看该图片时，标记就会被拉至图片，将鼠标悬停于代表图片区域的标记之上时就会变成高亮效果。已知用户曾使用过的标记，已用于相册的标记以及已连接到图片的标记，因此会有一个含有推荐使用标记的列表以供选取。（标记推荐系统还可以挂入外部系统以获取合适的标记）
• 不具图片批注权限的用户可能会在某张相片中认出某个人来，并希望有关此人的批注能够被创建。如批注的实际创建一样，用户针对某图片区域给出'建议'并标记该批注。然后具有足够权限的用户能够查看所有处理中的建议（甚至可以通过邮件告知的方式），接着针对这些建议作出取舍。同时，创建建议的用户可以对建议进行修改，直到建议被接受为止（那么对于匿名用户又如何呢？。

  如果建议被否决的话，创建该建议的用户会以某种形式接到通知，可以是查看建议时或是通过邮件告知。

• 不具足够权限的其他人如果看到某个图片标记不正确，就可以提出要求修改，然后具有足够权限的用户可以对修改请求作出决定。
• 某用户在某在线社交网络程序中结识了一些朋友，并希望在G2相片中为这些朋友进行标记，然后让社交网络程序（以及朋友）得知这一消息。
• 某用户希望搜索同一张图片中同时出现一组友人。
• 某用户希望搜索某友人面部占据图片大部分空间的图片。
• 某用户希望搜索朋友都不在眨眼时摄下的图片。

图片对象标记：

• 某用户希望对图片中某对象属性进行定义，以便搜索之用。

  使用单个标记想法的拓展。某标记定义某类别的一个实例，而该类别具有的一组属性可以针对某相片进行设定。举个例子，如果某人被标记了，该人可能正看着某个方向，面带微笑并眨着眼。该人可能会穿特定的衣服，而在图片中他就可能穿着这些衣服中的一件。

有关某图片区域的注释：

• 某用户希望使用有关图片特定区域的注释对图片进行批注。注释比简单的标记来说要更详尽，而且可以描述图片某区域的更多信息。当图片被查看时，用户就能看到一个区域被标记，并且在给予区域焦点（鼠标或键盘）时，该区域的相关注释就会显示出来。创建该批注的用户可以定位该批注并决定文本的显示方式--总是显示或图片区域给予焦点时才显示。
• 某用户希望搜索图片注释中具有特定文本的图片。

图片批注/涂鸦：

• 用户希望使用含有文本的聊天泡泡来为图片增色。用户可以选择让聊天泡泡总是显示或是在特定事件触发时才显示（比如鼠标移至某特定区域时）。而聊天泡泡自身也可以触发其他批注对象相互动的事件（这样就能显示一段聊天内容了）。
• 用户希望查找带有有趣内容的图片。

图片识别：

• 某图片识别程序搜索整个图片数据库并识别面孔及对象。该程序则对图片各类区域作出建议，并提交给具有足够权限查看这些图片的用户。具有权限的用户可以确认图片识别程序的分析并通知程序识别的正确与否；还可以为正确识别结果添加批注，并在识别错误时，告知程序图片区域的实际情况。

研究探讨

研究浏览器标签选取排序及渲染层：

YUI OverlayManager可以管理Overlay的z-order，但无法按照预期改变标签排序。浏览器标签排序以来DOM中的某个元素位置。让标签次序与层次序保持一致会比较好。

休要修改OverlayManager以对文档进行重新排序而不是操作z-index。SVG不能处理z-ordering，因此需要文档排序在正确的层被渲染。

其他实现图片区域批注的系统：

有不少论坛中提到的外部系统行为对图片区域批注来说很理想，因此有必要尝试进行实现，并使其在面对G2站点管理者/外观主题开发者时，能够正确发挥功能。相关研究谈到可以在此找到：其他系统的相关研究

设计

概览

• 保持可扩展性。

  希望该模块具有可扩展性，这样没有在此版本中出现的特点可以在将来加入。管理员也可仅启用想要的特点，从而减少页面载入时间，客户端操作时间以及资源占用。

概念

图片区域：

• 图片区域是图片上具有重要性的特定区域。该图片区域存在于同一张图片的所有版本之中（即源图片的所有派生）。注意派生图片的比例和定向可能与原始图片不同。
• 图片区域是由一个矩形DIV所表示的，也可以是是图片边界内的其他2d形状。

应可以将某个矩形DIV选择转为某个合适的2d形状。

• 所有图片区域无论形状都有一个2d划界的框架，这使得冲突检测更便捷。
• ?? 图片区域统筹在2d空间中显示为一个复杂的3维物体。图片区域可在2d空间中定义物体的3d属性，而不仅仅是显示一个2d区域，比如面孔，头部和身体的旋转。?? 这是否可以作为批注/其他关联？
• ?? 图片区域应能提供该区域的可缩放视图，以备页面中再次使用。

图片区域集：

• 一个图片区域集（image region set）是一组相关的图片区域。因为3d物体可以在2d空间中以不相交的区域表示出来，所以我们需能够将分离的图片区域链接在一起。（比如，当某人被标记了，而他的胳膊藏在某人背后，这种图片区域组合机制就变得重要了。我们标记整个图片区域集而不是分别标记各独立的2d图片区域）。

Image region hierarchy... does it make sense to have an image region made up of image region parts, and a part can be a parent to another image region part? Image region always has a root, which is the bounding DIV and then 1 or more children? Does it make sense to do this?

Annotation

• An annotation is something that is drawn in the browser (may not be visible due to style, but added into the DOM) and is associated with the image.
• ?? Does an image region need an annotation so that it can be displayed and interacted with? Or should we be able to attach style to an image region itself? If annotation required, need special circumstances to shape the annotation to the image region.
• The annotation, unlike an image region, is not necessarily desired on all sizes of the same image.
• An annotation may have a number of different style attributes depending on the derivative image (and Annotation set) it is being displayed with. Scale and rotation of the annotation may not be as simple as image regions, which are based purely on image co-ordinates.

e.g. A speech bubble annotation may not want to scale the same amount as the derivative image does.

• An annotation may be, but is not necessarily associated with an image region.

i.e. Tags and notes are associated with an image region, but a quirky speech bubble isn't.

• An annotation may be shown, hidden, or animated by some event.
• An annotation can be displayed in various positions:
  • Absolutely positioned, relative to the image.
  • Absolutely positioned, relative to another anotation.
  • Absolutely positioned, relative to the cursor.
  • In the imagenotes block area?
  • In a special annotation region for the annotation type.
  • In other special block areas of the correct type?

??Annotation parts

• An annotation may be made up of different parts and shapes and we may not want a database entry for every individual part. This may make the job of referencing different things more difficult however. And parts may want to be attached to different annotation regions. Examples:
  • Annotating a tag, we may want to display it at the cursor on mouse over, underneath the image (if a large group of people shown, a table of who's in the picture) and in the imagenotes block.
  • Quirky speech bubble animation contains multiple bubbles, animated together so when one bubble appears the other disappears. They together form a whole annotation, so may not want to be separated.

Annotation sets

• We may have lots of data about an image and not want to display all the data at the same time. Also, we may want different behaviours for interacting with the image when it is being viewed.
• An annotation belongs in one or more annotation sets.
• Annotations may have different properties per annotation set.
• Only one annotation set is viewable at a given time, otherwise different properties of an annotation being displayed may conflict. It should be possible to merge sets into one another, giving the opportunity to resolve conflicts.
• ?? Only one annotation set is rendered at a time (the others are not merely hidden, but don't exist in the DOM)?

Image region associations

• There may be reasons for creating associations with an image region that aren't visual markup. Not sure what yet though. :-)

Theming

• As with the rest of G2, site owners may want to make imagenotes appear in different ways. Though CSS styles may control a lot, there are instances where non-simple elements are needed. For instance, a selected area may want the inside of the area transparent, whilst not making the border transparent.

Imagenotes regions

There are a number of potentially desirable regions for use with Imagenotes, all with different purposes.

• Imagenotes block. When editing, all of the various image regions/annotations are shown here somehow. This allows you to interact with specific items potentially more easily than when editing over the image.
• Image area. The area within which the image is rendered.
• Sub image area. (Potentially another block?) Somewhere annotations such as who is in the image can be displayed.
• Annotation set selection. When viewing the image, something to choose which annotation set is being displayed (possible hidden when the user can only see one of the available sets).
• By the cursor. This is a special region that moves everywhere the cursor does. Potentially some logic is needed here for keeping this area on screen when the cursor is near the browser/frame border. Also, don't want to call code for moving the region when there is nothing being displayed in it.
• Toolbox. Buttons and/or labels for selection of viewing/editing modes and potentially all their sub toolkits.
• Docking areas (provided by themes or blocks?). Areas where toolboxes can be dragged to and inserted into the flow of the page.
• By an image region. When an image region is first created (or edited) and tags/notes are being applied, a convienient place to get the appropriate options is right next to the image region (or maybe just a modal dialog)

操作：

当进行添加/编辑时，以下操作可能是图片区域和批注都需要的。

• 拖动
  • 强制的
• 改变比例
  • 强制的
  • 宽高比
  • 最小值和最大值
• Z-ordering（分层）
• 组合/拆分
  • 多个组织中的图片区域？
  • 为形状设定操作（Union, Intersection, Negation）保留算符？
• 节点移动（当不为一简单矩形时，某形状的各点都可以单独移动）
  • 强制的。某形状是否会阻止某节点被移动到特定的位置以维持形状？
• 旋转
• 3d操作

模式：

特定模式（及子模式）会在图片交互中被用到。而模式则会决定行为。

• 查看。
• 编辑：
  • 编辑图片区域。
  • 编辑标记。
  • 编辑注释。
  • 编辑非图片区域的批注（涂鸦？）
• 用户建议（某不具任何关联的图片区域能够被建议？可能不行）：
  • 建议标记。
  • 建议注释。
  • 建议图画。
• 查看建议？查看形式可能与你编辑建议时的样式不同。
• 管理（建议回复）。
  • 显示谁/什么作出了建议。可以限制对某特定的提出建议者的查看。

Client side

Goals

• Make all actions responsive
  • Keep event handlers simple and quick.
  • Be wary of multiple super classes as call times may get out of hand.
  • Keep the item being displayed as simple as possible so there is less in the DOM. i.e. don't use templates for editing (with resize handles) when merely displaying.
  • ? Destroy classes when changing between modes or not? Memory consumed by keeping, CPU consumed by recreating.
• Minimise resource usage
• Minimise page load time
• Only load required features. Load features as required? (dynamically load JS then eval?)
• Degrade gracefully?

Notes

• This all really only works when javascript is enabled. It may be possible to render the markup server side without to atleast some degradeble functionality, but for now this is not being considered (would require the theme to put things in the right place).
• Image notes manager object handles all interactions with any given image (of course, within the scope of the imagenotes module) and is responsible for any page regions associated with the imagenotes (image, block area, toolkit)
• The image notes manager will be instanciated when the imagenotes block is included on photo pages.
• The image notes manager has to be pointed to the image it is managing. Currently a utility function searches out an <img> tag matching the item being displayed (this may not always work), with the image src URL passed in through the imagenotes block smarty template.
• The image notes manager is responsible for putting the image notes into the normal flow of browser focus, so when tabbing through the document editable regions and keyboard input works correctly.
• The image notes manager is responsible for client/server communication, plug in modules will communicate through it (is this bad design?, maybe it's more efficient for plugins to do their own thing to save unnecessary stuff being loaded, but structure the calls through an interface method).
• The image notes manager has a notion of 'mode'. The mode can be changed to allow editing of image regions, creation of annotations, and to generally change the view and behaviour of the imagenotes. An interface class will define the mode and other G2 modules will be able to add modes (collected through factory methods server side).
• The image notes manager will have rubber band functionality (just within the image area?) that is available to modes. Initially intended only for dragging out an image region, it may be useful for selection.
• Mode types, view, edit(/add/delete), suggest.
• Themes can define placement of toolbox, dock areas, annotation areas.
• A class will define an interface for a handler for imagenotes types. It handles a specific imagenote type. (e.g. for image tagging, the handler will look after the tag/image region relationship, even when an annotation doesn't exist)
• A class will define an interface for imagenotes types. An imagenote type is something that may be rendered to the page? and will probably fit into the imagenotes manager mode system. Will have events for it's selection, mouseover, etc.
• Can create annotation areas that fit into page flow and are not absolute(?) Can we determine this with javascript? Does it have to be done per item being displayed? Definitely theme specific. Will it be easily upset by adding other blocks? Sounds too complicated! :-)

Server side

Goals

• Minimise page load time, whilst being extensible

Notes

• Image region at client end should stick to co-ordinates on local image. Server side should convert co-ordinates to full size image co-ordinates for storage and to derivative size co-ordinates for client display.
• Only include javascript in page when necessary. Restrict to needed JS based on user permissions (i.e. edit code is not needed for someone that cannot edit)

Database structure

Goals

• Minimise page load time
• Design so it's easy and quick to search for interesting things

Notes

• Image region needs a unique ID, probably built from a key on the image it's attached to and it's own unique ID for the particular image. Maybe image region ID should have a single unique key to simplify cross referencing from other tables. Image region needs layer information to ensure the desired z-ordering is applied on rendering.

实现

客户端

• Created utility methods for creating instances of templates. That is, cloning a DOM node and applying IDs to the new node and it's children.
• Created TemplatedRubberBandRegion which allows a rubber band to be dragged over a specified region of the document and on mouseup a custom event is fired with the selected region
• Created BoxModelRegionOverlay which extends the YUI Overlay widget and allows placement of an overlay based on the margin, border, padding or content parts of the box model. This is so that image region overlays can be positioned and sized by the content area.
• Created TemplatedResizeOverlay which extends the BoxModelRegionOverlay widget and adds YUI DragDrop functionailty and the ability to resize the region. The code searches a template passed in client side for certain classes to turn into resize and drag handles. Custom events are fired at the start and end of drag and resize operations.
• Created an initial image region manager that sets up a TemplatedRubberBandRegion and listens for area selected events, then creates a TemplatedResizeOverlay. Multiple regions can be created, but only a single region can be focused at a time.
• The image region manager ensures that the image regions fit into the page focus flow.
• Created utility function for finding an <img> tag in the page with a specific src attribute to allow the image region manager to be attached.

服务器端

• Block created for addition to item view page
• No database structure created yet
• No server endpoint for AJAX communication created yet

调整或需要改良的区域

...

参考

G2特点请求

• (972082) 图片的标记区域
• (1053895) Imagemap注释
• (1741079) Fotonotes

论坛/Codex相关话题

• 如图片标签式的Facebook
• 图片注释

因为图片标签化为目的之一，一般标签的相关讨论可能也会令你感兴趣：

• Gallery模块：Tags
• TagsSearch 用于将标签分类

在一个图片中，不但具有查看位置，还能看到各类部分：

• Gallery模块：Map

外部系统

现存不少图片标记/注释的实例。一下例举一些：

• Fotonotes 在图片各类尺寸的区域中添加带有标题的注释。
• Flickr 在图片各类尺寸的区域中添加注释。
• Facebook 在图片固定尺寸的区域中添加标签（通常是用户的友人）。