Yang Shuqi

Textures

 

    Textures

    Image Edges ≠ Perceived Boundaries

    Perceived Boundaries rely on texture

    Objects vs Texture

    The same thing can be an object or a texture, depending on the scale of consideration

    Why Analyze Texture?

    For visual perception:

    • Indicative of material properties
    • Important appearance cue, especially if shape is similar across objects
    • Aim to distinguish between shape, boundaries, and texture

    For computer vision:

    • Representation-wise, we want a feature one step above basic building block of simple filters, edges.

    Texture Representation

    Example

    Use 2 filters to compute x- and y-derivatives as a 2-dimensional feature

    Draw (dx, dy) in the graph, the clusters represent different textures.

    Window Scale

    Possible to perform scale selection by looking for window scale where texture description not changing.

    Filter Banks

    Use multiple(d) filters to generate d-dimensional feature

    $D(a,b)=\sqrt{(a_1-b_1)^2+(a_2-b_2)^2}$

    A combination of scales and orientations, different types of patterns.

    Filter banks can measure redundant variety of structures in local neighborhood and result in multi-dimensional feature spaces

    When $a_1 \gg a_2$, $a_1$ dominates the distance D, and $a_2$ become meaningless. This is problematic if we need both features to separate our clusters. Normalizing the features fixes this.

    How many different filters? Implications?
    • Filters should “span” the range of textures we want to detect
    • This can be very compact set of orthogonal filters (e.g. x- & y-derivatives alone are sufficient), or it can be overcomplete and cover a whole range of orientations, patterns, etc.
    • The more similar the filters are, the more fine-tuned they are to respond to a specific orientation / pattern. This can be more discriminative for the application at hand (e.g. distinguishing v. similar textures)
    • More filters are not always better, since it
      • comes at the cost of a bigger vector, bigger computational effort and
      • may actual obfuscate the underlying pattern (think scenarios in which we want dimensionality reduction)
    • Limitation by filter size (i.e. 5x5 has more resolution than 3x3)

    Textures are Repeated Local Patterns

    Find the patterns
    • Use filters that look like patterns (spots, bars, raw patches…)
    • Consider magnitude of response
    Describe their statistics within each local window
    • Mean, standard deviation
    • Histogram
    • Histogram of “prototypical” feature occurrences
    Comparing Texton Histograms

    Naïve Choice: L2 / Euclidean

    • Euclidean distance is not ideal because it computes an arithmetic difference whereas the information reflected in histograms lies in relative values of their coordinates.

    • Better choices are:

      • χ2 (chi-square distance). Note that it is not symmetric, i.e. $d(h_1, h_2) ≠ d(h_2, h_1)$

        $d(H_1, H_2) = \sum_I {(H_1(I) - H_2(I))^2 \over H_1(I)}$

      • Bhattacharyya distance is a symmetric alternative to chi-square

        $d(H_1, H_2) = \sqrt {1-{1\over \sqrt {\bar H_1 \bar H_2 N^2}}\sum_I \sqrt{H_1(I) \cdot H_2(I)}}$

      • Histogram intersection

        $d(H_1, H_2) = \sum_Imin(H_1(I), H_2(I))$

    Textons

    An attempt to characterize texture

    Replace each pixel with integer representing the texture ‘type’

    Computing Textons

    Take a bank of filters and apply to lots of images

    For new pixel, filter surrounding region with same bank, and assign to nearest cluster; pixel now represented with the cluster id. The cluster id is the texton id.

    Histogram of Textons

    • Each material is now represented as a spatial arrangement of symbols from the texton vocabulary
    • Texture is defined by first order statistics of texton distribution, i.e., average density
    • For a given region, compute a histogram of textons as the representation: vector storing number of occurrences of each texton

    The smaller the difference between two histograms, the more similar they are.

    Texture representations via filter bank response “summarize” repeating patterns of local structure

    Histogram Models for Recognition
    • Every single count here comes from the response of a single pixel.
    • For each pixel for which we apply the entire filter bank, and derive a feature vector from the filter responses.
    • We check for the cluster to which it belongs and find the associated texton ID.
    • Sweep over all pixels to get the histogram which characterizes that region.

    Using Textons to Identify Boundaries

    Segmentation with texture features

    Find ‘textons’ by clustering vectors of filter bank outputs

    Describe texture in a window based on texton histogram

    What are Textures Good For?

    • Shape-from-texture
    • Classification
    • Retrieval
    • Segmentation
      • Analyze, represent texture
      • Group image regions with consistent texture
    • Synthesis
      • Generate new texture patches/images given some examples

    We can estimate surface’s 3D shape from deformation of texture from point to point

    Search