[
  {
    "name": "Conv",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "name": "pad"
        },
        {
          "default": 1,
          "name": "stride"
        },
        {
          "name": "exhaustive_search",
          "type": "boolean",
          "visible": false
        }
      ],
      "category": "Layer",
      "description": "\nThe convolution operator consumes an input vector, a filter blob\nand a bias blob and computes the output. \nThe Conv2D operator computes a 2D convolution operation over an input blob $(X)$, with a filter blob $(filter)$ and a bias blob $(bias)$, and outputs a single output blob $(Y)$. Although there are several options for order, the convention is that the input $(X)$ is a blob of shape $(N,C_{in},H_{in},W_{in})$ and the output $(Y)$ is a blob of shape $(N,C_{out},H_{out},W_{out})$. Here, $N$ is the batch size, $C$ is the number of channels, $H$ is the spatial height, and $W$ is the spatial width. For example, if your input data was a batch of five, 100x120pixel RGB images, $X$ would have shape $(5,3,120,100)$.\n\nThe $filter$ input blob may contain multiple filters and has shape $(M, C_{in}, K_H, K_W)$. Here, $M$ is the number of individual filters contained in the blob, $C_{in}$ is the number of channels of each filter (by convention in 2D convolution it is the same as the number of channels in the input), $K_H$ is the spatial height of the kernel, and $K_W$ is the spatial width of the kernel. The $bias$ blob is a vector of length $M$, where there is one bias for each filter in the $filter$ blob.\n\nGiven the shape of the input blob and the filter blob, we can calculate the shape of the output blob as follows. The number of items in the batch $N$ will stay the same. The number of channels in the output will equal the number of kernels in the filter blob, so $C_{out} = M.$ With stride and pad defined below, the spatial height and width of the output ($H_{out}$ and $W_{out}$) are calculated as\n\n$$H_{out} = \\left \\lfloor{\\frac{H_{in} - K_H + 2*pad}{stride}+1}\\right \\rfloor$$\n\n\n$$W_{out} = \\left \\lfloor{\\frac{W_{in} - K_W + 2*pad}{stride}+1}\\right \\rfloor$$\n\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Conv\",\n    [\"X\", \"filter\", \"bias\"],\n    [\"Y\"],\n    kernel=5,\n    pad=1,\n    stride=2\n)\n\n// Create X: (N,C,H,W)\ndata = np.random.randn(1,1,8,8).astype(np.float32)\nprint(\"Data shape: \",data.shape)\n\n// Create W: (M,C,Kh,Kw)\nfilters = np.random.randn(3,1,5,5).astype(np.float32)\nprint(\"Filter shape: \",filters.shape)\n\n// Create b: M\nbias = np.array([1.,1.,1.]).astype(np.float32)\nprint(\"Bias shape: \",bias.shape)\n\n// Put the inputs into the workspace\nworkspace.FeedBlob(\"X\", data)\nworkspace.FeedBlob(\"filter\", filters)\nworkspace.FeedBlob(\"bias\", bias)\n\n// Run the operator\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nData shape:  (1, 1, 8, 8)\nFilter shape:  (3, 1, 5, 5)\nBias shape:  (3,)\nY:\n [[[[  0.6406407    0.8620521    0.56461596]\n   [ -1.5042953   -0.79549205 -10.683343  ]\n   [ -0.5240259    3.4538248   -3.9564204 ]]\n\n  [[  0.6876496    4.8328524   -1.9525816 ]\n   [  1.2995434   -2.3895378    7.2670045 ]\n   [  3.9929862    1.8126237    5.4699917 ]]\n\n  [[  3.55949      4.7934155    0.76086235]\n   [  3.9588015   -1.3251319    4.413117  ]\n   [ -1.5296054   -1.4924102   -3.2552304 ]]]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "Input data blob, of shape $(N, C_{in}, H_{in}, W_{in})$, to be convolved with the kernels in the filter blob.",
          "name": "X"
        },
        {
          "description": "The filter blob, of shape $(M, C_{in}, K_H, K_W)$, containing the filters to be convolved with the data.",
          "name": "filter"
        },
        {
          "description": "The bias blob, of length $M$, containing the biases for the convolution, one bias per filter.",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob, of shape $(N, C_{out}, H_{out}, W_{out})$, that contains the result of the convolution.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ConvTranspose",
    "schema": {
      "attributes": [
        {
          "description": "Should the legacy padding be VALID or SAME. When used, pads should not be used.",
          "name": "legacy_pad",
          "option": "optional",
          "type": "int64"
        },
        {
          "description": "Desired kernel size. If left at default the kernel size will be inferred from the input $filter$ blob.",
          "name": "kernels",
          "option": "optional",
          "type": "int64[]"
        },
        {
          "description": "Controls the stride of the kernel as it traverses the input blob.",
          "name": "strides",
          "option": "optional",
          "type": "int64[]"
        },
        {
          "description": "Controls the amount of padding applied to the input feature map before computation.",
          "name": "pads",
          "option": "optional",
          "type": "int64[]"
        },
        {
          "description": "",
          "name": "adjs",
          "option": "optional",
          "type": "int64[]"
        },
        {
          "default": "NCHW",
          "description": "Specifies the order of the input data blob, where $N$ is batch size, $C$ is number of channels, $H$ is spatial height, and $W$ is spatial width. The only other valid option is \"NHWC\".",
          "name": "order",
          "option": "optional",
          "type": "string"
        },
        {
          "default": 0,
          "description": "",
          "name": "shared_buffer",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": false,
          "description": "",
          "name": "no_bias",
          "option": "optional",
          "type": "boolean"
        }
      ],
      "category": "Layer",
      "description": "\nThe ConvTranspose op takes an input data tensor $X$, an input weight tensor $filter$, and optionally an input bias tensor $bias$. It then computes the transposed convolution, sometimes referred to as deconvolution, and produces a single output tensor $Y$. The hyperparameters of the op such as kernel size, stride, and padding are specified as args. At each stride, the filter is deconvolved with a subset of $X$ and the $bias$ is added. This is done throughout the input data until the output computation is complete.\n\nThe output shapes are computed as follows. The number of channels in the output feature map is the number of kernels specified in the filter blob. The spatial height and width are computed as:\n\n$$H_{out} = (H_{in}-1)*strides[0] - 2*pads[0] + kernels[0]$$\n\n\n$$W_{out} = (W_{in}-1)*strides[1] - 2*pads[1] + kernels[1]$$\n\nNote on the implementation layout: conv_transpose_op_impl.h is the templated implementation of the conv_transpose_op.h file, which is why they are separate files. Also, in the implementation this operator inherits from the *ConvTransposeUnpoolOpBase* operator.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/tree/master/caffe2/operators/conv_transpose_op.h\n- https://github.com/pytorch/pytorch/tree/master/caffe2/operators/conv_transpose_op.cc\n- https://github.com/pytorch/pytorch/tree/master/caffe2/operators/conv_transpose_unpool_op_base.h\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ConvTranspose\",\n    [\"X\", \"filter\", \"bias\"],\n    [\"Y\"],\n    kernels=[2,2],\n    pads=[4,4,4,4],\n    strides=[2,2]\n)\n\n// Create X: (N,C,H,W)\ndata = np.random.randn(2,3,5,5).astype(np.float32)\nprint(\"Data shape: \",data.shape)\n\n// Create filter: (M,C,Kh,Kw)\nfilters = np.random.randn(3,1,2,2).astype(np.float32)\nprint(\"Filter shape: \",filters.shape)\n\n// Create b: M\nbias = np.array([1.]).astype(np.float32)\nprint(\"Bias shape: \",bias.shape)\n\n// Put the inputs into the workspace\nworkspace.FeedBlob(\"X\", data)\nworkspace.FeedBlob(\"filter\", filters)\nworkspace.FeedBlob(\"bias\", bias)\n\n// Run the operator\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nData shape:  (2, 3, 5, 5)\nFilter shape:  (3, 1, 2, 2)\nBias shape:  (1,)\nY:\n [[[[0.53606427 0.5775447 ]\n   [0.40148795 1.5188271 ]]]\n\n\n [[[1.9903406  3.2794335 ]\n   [0.09960175 0.31917763]]]]\n\n```\n\n</details>\n\n  ",
      "inputs": [
        {
          "description": "Input data blob, of shape $(N, C_{in}, H_{in}, W_{in})$, to be operated on.",
          "name": "X"
        },
        {
          "description": "The filter blob, of shape $(M, C_{out}, K_H, K_W)$, containing the filters to be used in the transposed convolution.",
          "name": "filter"
        },
        {
          "description": "The bias blob, of length $C_{out}$, containing the biases for the operation, one bias per output channel. If not passed, biases assumed to be zeros.",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob, of shape $(N, C_{out}, H_{out}, W_{out})$, that contains the result of the operation.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FC",
    "schema": {
      "attributes": [
        {
          "default": 1,
          "description": "Describes the axis of the input data $X$. Defaults to one because in the common case when the input $X$ has shape $(M,K)$, the first axis encodes the batch size.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": 1,
          "description": "Describes the axis of the input weight matrix $W$. Defaults to one because the first axis most likely describes the batch_size.",
          "name": "axis_w",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": false,
          "description": "Whether to use float-16 compute kernel.",
          "name": "float16_compute",
          "option": "optional",
          "type": "boolean"
        }
      ],
      "category": "Layer",
      "description": "\nThe FC operator computes an output $(Y)$ as a linear combination of the input data blob $(X)$ with a weight blob $(W)$ and bias blob $(b)$. More formally,\n\n$$Y = XW^T+b$$\n\nHere, $X$ is a matrix of shape $(M,K)$, $W$ is a matrix of shape $(N,K)$, $b$ is a vector of length $N$, and $Y$ is a matrix of shape $(M,N)$. $N$ can be thought of as the number of nodes in the layer, $M$ is the batch size, and $K$ is the number of features in an input observation.\n\n*NOTE: $X$ does not need to explicitly be a 2-dimensional matrix, however, if it is not it will be coerced into one. For an arbitrary $n$-dimensional tensor $X$, e.g. $[a_0, a_1, \\ldots ,a_{k-1}, a_k, \\ldots , a_{n-1}]$, where $a_i$ in $N$, and $k$ is the $axis$ arg provided, then $X$ will be coerced into a 2-dimensional tensor with dimensions $[a_0 * \\ldots * a_{k-1}, a_k * \\ldots * a_{n-1}]$. For the default case where axis=1, this means the $X$ tensor will be coerced into a 2D tensor of dimensions $[a_0, a_1 * \\ldots * a_{n-1}]$, where $a_0$ is often the batch size. In this situation, we must have $a_0 = M$ and $a_1 * \\ldots * a_{n-1} = K$. Lastly, even though $b$ is a vector of length $N$, it is copied and resized to shape $(M x N)$ implicitly, then added to each vector in the batch.*\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/fully_connected_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/fully_connected_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\n// In this example, our batch size is 1 (M=1), the input observation will have\n//   6 features (K=6), and the layer will have one hidden node (N=1). The\n//   expected output is Y=7.\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"FC\",\n    [\"X\", \"W\", \"b\"],\n    [\"Y\"]\n)\n\n// Create X: MxK\ndata = np.array([1,2,3,4,5,6]).astype(np.float32)\ndata = data[np.newaxis,:]\n\n// Create W: NxK\nweights = np.array(np.array([1,1/2.,1/3.,1/4.,1/5.,1/6.])).astype(np.float32)\nweights = weights[np.newaxis,:]\n\n// Create b: N\nbias = np.array([1.]).astype(np.float32)\n\n// Put the inputs into the workspace\nworkspace.FeedBlob(\"X\", data)\nworkspace.FeedBlob(\"W\", weights)\nworkspace.FeedBlob(\"b\", bias)\n\n// Run the operator\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nY:\n [[7.]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "Input blob to be coerced into a 2D matrix of shape $(M,K)$, where $M$ is the batch size and $K$ is the number of features in a single observation.",
          "name": "X"
        },
        {
          "description": "Input blob to be coerced into a 2D matrix of shape $(N,K)$ describing a fully connected weight matrix. Here, $K$ is the number of features in a single observation and $N$ is the number of nodes in the FC layer.",
          "name": "W"
        },
        {
          "description": "Input blob containing vector of length $N$ which describes one bias for each node in the layer.",
          "name": "b"
        }
      ],
      "outputs": [
        {
          "description": "Output blob containing a 2D output matrix of shape $(M,N)$, where $M$ is the batch size and $N$ is the number of nodes in the layer. The output is calculated as $Y=XW^T+b$.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Add",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise binary addition (with limited broadcast support).\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Add\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", np.array([[1,2],[3,4]]))\nworkspace.FeedBlob(\"B\", np.array([[5,6],[7,8]]))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\n\nA:\n[[1 2]\n [3 4]]\nB:\n[[5 6]\n [7 8]]\nC:\n[[ 6  8]\n [10 12]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<float>`)* Second operand. With broadcasting can be of smaller size than A. If broadcasting is disabled it should be of the same size as A.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor with same dimensions and type as A.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Sum",
    "schema": {
      "description": "\nElement-wise sum of each of the input tensors. The first input tensor can be used\nin-place as the output tensor, in which case the sum will be done in place and\nresults will be accumulated the first input tensor. All inputs and outputs must\nhave the same shape and data type.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_sum_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Sum\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", np.array([[1,2],[3,4]]).astype(np.float32))\nworkspace.FeedBlob(\"B\", np.array([[5,6],[7,8]]).astype(np.float32))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"A\"))\n\n```\n\n**Result**\n\n```\n\nA: [[1. 2.]\n [3. 4.]]\nB: [[5. 6.]\n [7. 8.]]\nC: [[1. 2.]\n [3. 4.]]\n\n```\n\n</details>\n\n<details>\n\n<summary> <b>Example 2</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Sum\",\n    [\"A\",  \"B\"],\n    [\"A\"],  // inplace\n)\n\nworkspace.FeedBlob(\"A\", np.array([[1,2,5],[8,3,4]]).astype(np.float32))\nworkspace.FeedBlob(\"B\", np.array([[9,5,6],[6,7,8]]).astype(np.float32))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"A after Sum:\", workspace.FetchBlob(\"A\"))\n\n```\n\n**Result**\n\n```\n\nA: [[1. 2. 5.]\n [8. 3. 4.]]\nB: [[9. 5. 6.]\n [6. 7. 8.]]\nA after Sum: [[10.  7. 11.]\n [14. 10. 12.]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* First tensor to be added element-wise.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<float>`)* Second tensor to be added element-wise.",
          "name": "B"
        },
        {
          "description": "First of the input tensors. Can be inplace.",
          "name": "data_0"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Sum of A and B.",
          "name": "C"
        },
        {
          "description": "Output tensor. Same dimension as inputs.",
          "name": "sum"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Mul",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise binary multiplication (with limited broadcast support).\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Mul\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", np.array([[1,2],[3,4]]))\nworkspace.FeedBlob(\"B\", np.array([[5,6],[7,8]]))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\n\nA:\n[[1 2]\n [3 4]]\nB:\n[[5 6]\n [7 8]]\nC:\n[[ 5 12]\n [21 32]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<float>`)* Second operand. With broadcasting can be of smaller size than A. If broadcasting is disabled it should be of the same size as A.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor with same dimensions and type as A.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MatMul",
    "schema": {
      "attributes": [
        {
          "default": 1,
          "description": "Exclusive axis that divides the first and second dimension of matrix `A`.",
          "name": "axis_a",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": 1,
          "description": "Exclusive axis that divides the first and second dimension of matrix `B`.",
          "name": "axis_b",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": 0,
          "description": "Pass 1 to transpose `A` before multiplication and after the dimension adjustment using `axis_a`.",
          "name": "trans_a",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": 0,
          "description": "Pass 1 to transpose `B` before multiplication and after the dimension adjustment using `axis_b`.",
          "name": "trans_b",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nMatrix multiplication $Y = A * B$, where `A` has size (M x K), `B` has size\n(K x N), and `Y` will have a size (M x N). To transpose `A` or `B` before\nmultiplication, pass 1 to the `trans_a` and/or `trans_b` arguments, which\nseparate the first and second dimensions of the respective matrices using\n`axis_a` and `axis_b`.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/matmul_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"MatMul\",\n    [\"A\", \"B\"],\n    [\"Y\"],\n)\n\nworkspace.FeedBlob(\"A\", np.random.randint(10, size=(3,3)).astype(np.float32))\nworkspace.FeedBlob(\"B\", np.random.randint(10, size=(3,3)).astype(np.float32))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nA: [[1. 8. 3.]\n [6. 4. 4.]\n [5. 4. 7.]]\nB: [[4. 0. 3.]\n [3. 1. 1.]\n [8. 5. 8.]]\nY: [[52. 23. 35.]\n [68. 24. 54.]\n [88. 39. 75.]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* 2D matrix of size (M x K).",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<float>`)* 2D matrix of size (K x N).",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* 2D matrix of size (M x N).",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Relu",
    "schema": {
      "attributes": [
        {
          "name": "cudnn_exhaustive_search",
          "type": "boolean",
          "visible": false
        }
      ],
      "category": "Activation",
      "description": "\nApplies rectified linear unit operation to the input data element-wise. The Relu operation takes one input $X$, produces one output $Y$, and is defined as:\n\n$$Y = max(0,X)$$\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/relu_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/relu_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n  \"Relu\",\n  [\"X\"],\n  [\"Y\"]\n  )\n\nworkspace.FeedBlob(\"X\", np.random.randn(4, 4).astype(np.float32)) // NCHW\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\n\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[-1.4655551   0.64575136  0.7921748   0.4150579 ]\n [ 0.41085166 -0.2837964   0.9881425  -1.9300346 ]\n [ 0.39705405  0.44639114  0.9940703   0.2926532 ]\n [-0.6726489   0.01330667  1.101319    0.33858967]]\n\nY:\n [[0.         0.64575136 0.7921748  0.4150579 ]\n [0.41085166 0.         0.9881425  0.        ]\n [0.39705405 0.44639114 0.9940703  0.2926532 ]\n [0.         0.01330667 1.101319   0.33858967]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "1D input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "1D output tensor with same shape as input",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Sigmoid",
    "schema": {
      "category": "Activation",
      "description": "\nApply the Sigmoid function element-wise to the input tensor. This is often used\nas a non-linear activation function in a neural network. The sigmoid function is\ndefined as:\n\n$$Sigmoid(x) = \\frac{1}{1+\\exp(-x)}$$\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/sigmoid_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Sigmoid\",\n    [\"X\"],\n    [\"Y\"]\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(5).astype(np.float32))\nprint(\"input:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"sigmoid:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\ninput: [ 1.5744036   0.31632107  1.7842269   1.4450722  -2.1726978 ]\nsigmoid: [0.8284105  0.57842743 0.85621804 0.80923885 0.10222916]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PRelu",
    "schema": {
      "category": "Activation",
      "description": "\n\nThe *PRelu* op takes input data tensor $X$, an input slope tensor $slope$, and produces one output tensor $Y$ of the same shape as $X.$ The op performs the element wise *PRelu* operation, defined as\n\n$$y=prelu(x) =\\begin{cases}slope * x & x < 0\\\\x & otherwise\\end{cases}$$\n\nNote, is slope is size 1, the value is shared across the channels, otherwise $X$ and $slope$ must be the same shape. See [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://arxiv.org/abs/1502.01852) for more information.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/prelu_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/prelu_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"PRelu\",\n    [\"X\",\"Slope\"],\n    [\"Y\"],\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(3, 3).astype(np.float32))\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\n\nworkspace.FeedBlob(\"Slope\", np.array([0.1]).astype(np.float32))\nprint(\"Slope:\\n\", workspace.FetchBlob(\"Slope\"), \"\\n\")\n\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[ 0.3957382  -0.19725518 -0.26991343]\n [ 1.5513182  -0.27427664 -0.14584002]\n [-0.4121164   0.9292345   0.96426094]]\n\nSlope:\n [0.1]\n\nY:\n [[ 0.3957382  -0.01972552 -0.02699134]\n [ 1.5513182  -0.02742766 -0.014584  ]\n [-0.04121164  0.9292345   0.96426094]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "Input tensor of data to be operated on.",
          "name": "X"
        },
        {
          "description": "1D input slope tensor. If `Slope` is of size 1, the value is shared across different channels",
          "name": "Slope"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor, with same shape as $X$.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Softmax",
    "schema": {
      "attributes": [
        {
          "default": 1,
          "description": "Axis of the inputs when coerced to 2D matrix.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "category": "Activation",
      "description": "\n\nApplies the Softmax function to an n-dimensional input Tensor rescaling them so\nthat the elements of the n-dimensional output Tensor lie in the range (0,1) and\nsum to 1. The softmax operator is typically the last layer in a classifier network,\nas its output can be interpreted as confidence probabilities of an input belonging\nto each class. The input is a 2-D tensor (Tensor) of size (batch_size x\ninput_feature_dimensions). The output tensor has the same shape and contains the\nsoftmax normalized values of the corresponding input. The softmax function is\ndefined as follows:\n\n$$softmax(x_i) = \\frac{\\exp(x_i)}{\\sum_{j} \\exp(x_j)}$$\n\nThe input does not need to explicitly be a 2D vector; rather, it will be coerced\ninto one. For an arbitrary n-dimensional tensor `X` in\n$[a_0, a_1, ..., a_{k-1}, a_k, ..., a_{n-1}]$, where k is the `axis` provided,\nthen `X` will be coerced into a 2-dimensional tensor with dimensions\n$[(a_0 * ... * a_{k-1}), (a_k * ... * a_{n-1})]$. For the default case where\n`axis`=1, the `X` tensor will be coerced into a 2D tensor of dimensions\n$[a_0, (a_1 * ... * a_{n-1})]$, where $a_0$ is often the batch size. In this\nsituation, we must have $a_0 = N$ and $a_1 * ... * a_{n-1} = D$. Each of these\ndimensions must be matched correctly, or else the operator will throw errors.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/softmax_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/softmax_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Softmax\",\n    [\"X\"],\n    [\"Y\"]\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(1, 5).astype(np.float32))\nprint(\"input:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"softmax:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\ninput: [[ 0.0417839   0.61960053 -0.23150268 -0.64389366 -3.0000346 ]]\nsoftmax: [[0.24422921 0.43525138 0.18582782 0.12303016 0.01166145]]\n\n```\n\n</details>\n\n\n\n",
      "inputs": [
        {
          "description": "The input tensor that's coerced into a 2D matrix of size (NxD) as described above.",
          "name": "input"
        },
        {
          "description": "*(type: Tensor`<float>`)* Input tensor that's coerced into a 2D matrix of size (NxD) as described above.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "The softmax normalized output values with the same shape as input tensor.",
          "name": "output"
        },
        {
          "description": "*(type: Tensor`<float>`)* The softmax normalized output tensor with the same shape as input tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MaxPool",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "name": "order"
        },
        {
          "default": 0,
          "name": "pad"
        },
        {
          "name": "cudnn_exhaustive_search",
          "type": "boolean",
          "visible": false
        }
      ],
      "category": "Pool",
      "description": "MaxPool \nconsumes an input blob and applies max pooling across the the blob according to\nkernel sizes, stride sizes, pad lengths and dilation. Max pooling consists of\ntaking the maximum value of a subset of the input tensor according to the kernel\nsize and downsampling the data into the output blob for further processing. The\n`brew` module has a wrapper for this operator for use in a `ModelHelper` object.\n\nPooling layers reduce the spatial dimensionality of the input blob. Each of the\noutput blob's dimensions will reduce according to:\n\n$$dim_{out}=\\frac{dim_{in}-kernel+2*pad}{stride}+1$$\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"MaxPool\",\n    [\"X\"],\n    [\"Y\"],\n    kernel=2,\n    stride=2,\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(1, 1, 6, 6).astype(np.float32)) // NCHW\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX:\n [[[[-2.8534958e-01 -1.7719941e+00 -8.2277227e-04  1.1088650e+00\n    -2.1476576e+00 -3.5070452e-01]\n   [-9.0058845e-01 -3.0070004e-01 -1.7907504e+00 -7.1746534e-01\n     1.2798511e+00 -3.2214901e-01]\n   [ 1.5806322e+00  1.6845188e+00 -2.6633200e-01 -3.8576153e-01\n    -9.6424848e-02 -3.9696163e-01]\n   [ 1.2572408e-01  6.3612902e-01 -3.9554062e-01 -6.9735396e-01\n    -9.1898698e-01 -1.9609968e-01]\n   [-1.1587460e+00  2.4605224e+00 -1.5497679e+00  1.3020347e-01\n    -8.1293899e-01 -7.8803545e-01]\n   [ 1.4323474e+00  1.3618395e+00  9.8975077e-02 -1.1307785e-01\n     7.2035044e-01  2.7642491e-01]]]]\n\nY:\n [[[[-0.28534958  1.108865    1.2798511 ]\n   [ 1.6845188  -0.266332   -0.09642485]\n   [ 2.4605224   0.13020347  0.72035044]]]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor of shape NCHW or NHWC.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output data tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AveragePool",
    "schema": {
      "category": "Pool",
      "description": "AveragePool \nconsumes an input blob and applies average pooling across the the blob according\nto kernel sizes, stride sizes, pad lengths and dilation. Average pooling consists\nof taking the average value of a subset of the input tensor according to the kernel\nsize and downsampling the data into the output blob for further processing. The\n`brew` module has a wrapper for this operator for use in a `ModelHelper` object.\n\nPooling layers reduce the spatial dimensionality of the input blob. Each of the\noutput blob's dimensions will reduce according to:\n\n$$dim_{out}=\\frac{dim_{in}-kernel+2*pad}{stride}+1$$\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"AveragePool\",\n    [\"X\"],\n    [\"Y\"],\n    kernel=2,\n    stride=2,\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(1, 1, 6, 6).astype(np.float32)) // NCHW\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX:\n [[[[-0.2883434   0.43498734  0.05417408  1.912558    0.09390241\n    -0.33173105]\n   [ 1.633709    1.2047161   0.36964908  0.99961185  0.4184147\n     0.9989975 ]\n   [ 1.7644193   0.1789665   1.5812988  -0.6038542  -0.36090398\n     0.33195344]\n   [ 0.9457722  -0.95174325 -0.78124577  1.2062047   1.1903144\n     0.2586746 ]\n   [ 1.252104    0.32645547  1.8073524  -0.78397465  0.9978303\n    -0.97614396]\n   [ 0.5440196   1.5778259  -0.76750124  0.5051756   0.8838398\n    -0.37085298]]]]\n\nY:\n [[[[0.7462672  0.83399826 0.2948959 ]\n   [0.4843537  0.3506009  0.35500962]\n   [0.9251013  0.19026303 0.13366827]]]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor of shape NCHW or NHWC.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output data tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SpatialBN",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "If set to nonzero, run spatial batch normalization in test mode.",
          "name": "is_test",
          "type": "int64"
        },
        {
          "default": 1e-05,
          "description": "The epsilon value to use to avoid division by zero.",
          "name": "epsilon",
          "option": "optional",
          "type": "float32"
        },
        {
          "default": "NCHW",
          "description": "Specifies the order of the input data blob, where $N$ is batch size, $C$ is number of channels, $H$ is spatial height, and $W$ is spatial width. The only other valid option is \"NHWC\".",
          "name": "order",
          "option": "optional",
          "type": "string"
        },
        {
          "default": 0.9,
          "description": "Factor used in computing the running mean and variance. e.g., running_mean = running_mean x momentum + mean x (1 - momentum)",
          "name": "momentum",
          "option": "optional",
          "type": "float32"
        },
        {
          "default": 1,
          "description": "Specifies the number of batches to apply normalization on. Requires specifying the optional sums and sumsq inputs that provide statistics across multiple batches from which mean and variance can be determined.",
          "name": "num_batches",
          "option": "optional",
          "type": "int64"
        }
      ],
      "category": "Normalization",
      "description": "\nApplies spatial batch normalization to the input tensor as described in the original paper, [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://arxiv.org/abs/1502.03167). Be aware, this operator has two different output sets, depending on the value of *is_test*. According to the paper, the primary operation of spatial batch normalization is:\n\n$$Y = \\frac{X - \\mu_x}{\\sqrt{\\sigma^2_{x} + \\epsilon}}*\\gamma + b$$\n\nIn the equation, $\\mu_x$ is the *mean*, $X$ is the input data, $\\sigma^2_{x}$ is the *var*, $\\epsilon$ is *epsilon*, $\\gamma$ is the *scale*, $b$ is the *bias*, and $Y$ is the output data. The *momentum* arg also affects this calculation in the computation of the running mean and variance. The influence of *momentum* is as follows:\n\n$$running\\_mean = running\\_mean * momentum + mean * (1 - momentum)$$\n\n$$running\\_var = running\\_var * momentum + var * (1 - momentum)$$\n\nOutput when is_test = 0 (train mode): *Y, mean, var, saved_mean, saved_var*\n\nOutput when is_test = 1 (test mode): *Y*\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/spatial_batch_norm_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/spatial_batch_norm_op.h\n\n",
      "inputs": [
        {
          "name": "input"
        },
        {
          "description": "The scale as a 1-dimensional tensor of size $C$ to be applied to the output.",
          "name": "scale"
        },
        {
          "description": "The bias as a 1-dimensional tensor of size $C$ to be applied to the output.",
          "name": "bias"
        },
        {
          "description": "The running mean (training) or the estimated mean (testing) as a 1-dimensional tensor of size $C$.",
          "name": "mean"
        },
        {
          "description": "The running variance (training) or the estimated variance (testing) as a 1-dimensional tensor of size $C$.",
          "name": "var"
        },
        {
          "description": "The input 4-dimensional tensor of shape $NCHW$ or $NHWC$ depending on the order parameter.",
          "name": "X"
        },
        {
          "description": "*(optional)* Per-channel sums of elements to be used to determine the mean and variance for this batch.",
          "name": "sums"
        },
        {
          "description": "*(optional)* Per-channel sum of elements squared per channel to be used to determine the variance for this batch.",
          "name": "sumsq"
        }
      ],
      "outputs": [
        {
          "description": "The output 4-dimensional tensor of the same shape as $X$.",
          "name": "Y"
        },
        {
          "description": "The running mean after the spatial BN operator. Must be in-place with the input *mean*. Should not be used for testing.",
          "name": "mean"
        },
        {
          "description": "The running variance after the spatial BN operator. Must be in-place with the input *var*. Should not be used for testing.",
          "name": "var"
        },
        {
          "description": "Saved mean used during training to speed up gradient computation. Should not be used for testing.",
          "name": "saved_mean"
        },
        {
          "description": "Saved variance used during training to speed up gradient computation. Should not be used for testing.",
          "name": "saved_var"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LRN",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Amount of neighboring channels to sum over for normalization",
          "name": "size",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": 0.0,
          "description": "Multiplicative (scaling) factor.",
          "name": "alpha",
          "option": "optional",
          "type": "float32"
        },
        {
          "default": 0.0,
          "description": "Exponent.",
          "name": "beta",
          "option": "optional",
          "type": "float32"
        },
        {
          "default": 1.0,
          "description": "Additive factor.",
          "name": "bias",
          "option": "optional",
          "type": "float32"
        },
        {
          "default": 0,
          "description": "Order of blob dimensions.",
          "name": "order",
          "option": "optional",
          "type": "float32"
        }
      ],
      "category": "Normalization",
      "description": "\n\n`LRN` applies Local Response Normalization to an input blob. This operation performs\na kind of \"lateral inhibition\" by normalizing over local input regions, where\nnormalization is applied across channels. This operator is typically used to\nnormalize an unbounded activation (such as ReLU). The output shape is the same as\nthe input shape. The `brew` module has a wrapper for this operator for use in a\n`ModelHelper` object.\n\nThe formula for LRN is as follows:\n\n$$b_{c} = a_{c}(bias + \\frac{\\alpha}{n}\\sum_{c'=max(0,c-n/2)}^{min(N-1,c+n/2)} a_{c'}^2 )^{-\\beta}$$\n\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/local_response_normalization_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/local_response_normalization_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\"LRN\",\n     [\"X\"],\n     [\"Y\", \"Y_scale\"],\n     size=11,\n     alpha=0.001,\n     beta=0.5,\n     bias=2.0,\n     order=\"NHWC\"\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(1, 6, 6, 1).astype(np.float32)) // NCHW\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\nprint(\"Y_scale:\\n\", workspace.FetchBlob(\"Y_scale\"))\n```\n\n**Result**\n\n```\nX:\n [[[[ 0.72985137]\n   [-0.3753357 ]\n   [ 2.7344604 ]\n   [-0.5937792 ]\n   [ 0.38440478]\n   [-2.1659644 ]]\n\n  [[-0.92846817]\n   [-0.9996144 ]\n   [ 0.212943  ]\n   [-1.968045  ]\n   [-0.77839696]\n   [ 0.45492038]]\n\n  [[-0.11263168]\n   [ 1.9901097 ]\n   [ 0.19275683]\n   [ 0.15630436]\n   [ 0.7536298 ]\n   [-0.77339894]]\n\n  [[ 0.8353551 ]\n   [-0.7784452 ]\n   [ 1.779317  ]\n   [ 0.22421335]\n   [ 1.3846219 ]\n   [-3.0546608 ]]\n\n  [[ 0.09977621]\n   [ 2.2071757 ]\n   [ 0.79971045]\n   [ 3.563886  ]\n   [-0.7169287 ]\n   [ 0.77170426]]\n\n  [[-1.4296649 ]\n   [ 0.19181213]\n   [ 0.45961624]\n   [-1.0201577 ]\n   [ 0.62854475]\n   [-0.6395456 ]]]]\n\nY:\n [[[[ 0.5160766 ]\n   [-0.26540157]\n   [ 1.9332271 ]\n   [-0.41986194]\n   [ 0.27181432]\n   [-1.5314047 ]]\n\n  [[-0.6565133 ]\n   [-0.7068181 ]\n   [ 0.15057328]\n   [-1.3914955 ]\n   [-0.5504022 ]\n   [ 0.32167578]]\n\n  [[-0.0796426 ]\n   [ 1.4070934 ]\n   [ 0.13629955]\n   [ 0.11052381]\n   [ 0.53288984]\n   [-0.5468682 ]]\n\n  [[ 0.5906759 ]\n   [-0.5504363 ]\n   [ 1.2580767 ]\n   [ 0.1585426 ]\n   [ 0.9790328 ]\n   [-2.1595135 ]]\n\n  [[ 0.07055242]\n   [ 1.5605361 ]\n   [ 0.5654725 ]\n   [ 2.5193207 ]\n   [-0.50693923]\n   [ 0.54567   ]]\n\n  [[-1.0108787 ]\n   [ 0.13563155]\n   [ 0.3249962 ]\n   [-0.72134334]\n   [ 0.44444424]\n   [-0.45222285]]]]\nY_scale:\n [[[[2.0000484]\n   [2.0000129]\n   [2.0006797]\n   [2.000032 ]\n   [2.0000134]\n   [2.0004265]]\n\n  [[2.0000784]\n   [2.0000908]\n   [2.000004 ]\n   [2.0003521]\n   [2.000055 ]\n   [2.0000188]]\n\n  [[2.0000012]\n   [2.00036  ]\n   [2.0000033]\n   [2.0000021]\n   [2.0000517]\n   [2.0000544]]\n\n  [[2.0000634]\n   [2.000055 ]\n   [2.0002878]\n   [2.0000045]\n   [2.0001743]\n   [2.0008483]]\n\n  [[2.000001 ]\n   [2.000443 ]\n   [2.0000582]\n   [2.0011547]\n   [2.0000467]\n   [2.0000541]]\n\n  [[2.0001857]\n   [2.0000033]\n   [2.0000193]\n   [2.0000947]\n   [2.000036 ]\n   [2.0000372]]]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor (ReLU output).",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor.",
          "name": "Y"
        },
        {
          "description": "*(type: Tensor`<float>`)* Output scale.",
          "name": "Y_scale"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Dropout",
    "schema": {
      "attributes": [
        {
          "default": 0.5,
          "description": "Probability of an element to be zeroed.",
          "name": "ratio",
          "option": "optional",
          "type": "float32"
        },
        {
          "default": 0,
          "description": "If zero (train mode), perform dropout. If non-zero(test mode), Y = X.",
          "name": "is_test",
          "type": "int64"
        }
      ],
      "category": "Dropout",
      "description": "\n\n`Dropout` takes one input data tensor (`X`) and produces two tensor outputs, `Y` and\n`mask`. If the `is_test` argument is zero (default=0), the output `Y` will be the input\nwith random elements zeroed. The probability that a given element is zeroed is\ndetermined by the `ratio` argument.\n\nIf the `is_test` argument is set to non-zero, the output `Y` is exactly the same as the\ninput `X`. Note that outputs are scaled by a factor of $\\frac{1}{1-ratio}$ during\ntraining, so that during test time, we can simply compute an identity function. This\nscaling is important because we want the output at test time to equal the expected value\nat training time. Dropout has been proven to be an effective regularization technique to\nprevent overfitting during training.\n\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/dropout_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/dropout_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Dropout\",\n    [\"X\"],\n    [\"Y\"] + [\"mask\"],\n    ratio=0.5,\n    is_test=0\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(5, 5)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\nprint(\"mask:\", workspace.FetchBlob(\"mask\"))\n```\n\n**Result**\n\n```\nX: [[5. 4. 3. 6. 9.]\n [2. 1. 8. 0. 9.]\n [7. 3. 0. 6. 3.]\n [1. 8. 2. 6. 4.]\n [6. 2. 6. 4. 0.]]\nY: [[ 0.  0.  0. 12. 18.]\n [ 0.  0. 16.  0.  0.]\n [ 0.  0.  0. 12.  6.]\n [ 0.  0.  4.  0.  0.]\n [12.  0.  0.  0.  0.]]\nmask: [[False False False  True  True]\n [False False  True  True False]\n [False False  True  True  True]\n [False False  True False False]\n [ True False False False False]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "The input data as Tensor.",
          "name": "data"
        },
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "The output.",
          "name": "output"
        },
        {
          "description": "*(type: Tensor`<bool>`)* The output mask containing boolean values foreach element, signifying which elements are dropped out. If `is_test` isnonzero, this output is not filled.",
          "name": "mask"
        },
        {
          "description": "*(type: Tensor`<float>`)* Output tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Concat",
    "schema": {
      "attributes": [
        {
          "default": -1,
          "description": "Axis to concatenate on.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        },
        {
          "description": "Order of blob dimensions. Concats on the C dimension.",
          "name": "order",
          "option": "optional",
          "type": "string"
        },
        {
          "description": "Pass non-zero integer to add the axis specified in `axis` to all input tensors.",
          "name": "add_axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "category": "Tensor",
      "description": "\nConcatenate a list of tensors into a single tensor. Similar functionality to\nNumpy's [concatenate](https://docs.scipy.org/doc/numpy/reference/generated/numpy.concatenate.html)\nfunction. The `axis` argument specifies what axis along which the arrays will be concatenated.\nWhen set to non-zero (default=0), the `add_axis` argument adds the axis specified in `axis` to\nall input tensors.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/concat_split_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/concat_split_op.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Concat\",\n    [\"X1\",  \"X2\"],\n    [\"Y\", \"split_info\"],\n    axis=0\n)\n\nworkspace.FeedBlob(\"X1\", np.array([[1,2],[3,4]]))\nworkspace.FeedBlob(\"X2\", np.array([[5,6]]))\nprint(\"X1:\", workspace.FetchBlob(\"X1\"))\nprint(\"X2:\", workspace.FetchBlob(\"X2\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\nprint(\"split_info:\", workspace.FetchBlob(\"split_info\"))\n\n```\n\n**Result**\n\n```\n\nX1: [[1 2]\n [3 4]]\nX2: [[5 6]]\nY: [[1 2]\n [3 4]\n [5 6]]\nsplit_info: [2 1]\n\n```\n\n</details>\n\n<details>\n\n<summary> <b>Example 2</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Concat\",\n    [\"X1\",  \"X2\"],\n    [\"Y\", \"split_info\"],\n    add_axis=1,\n    axis=3\n)\n\nworkspace.FeedBlob(\"X1\", np.random.randint(10, size=(1, 1, 5, 5))) // NCHW\nworkspace.FeedBlob(\"X2\", np.random.randint(10, size=(1, 1, 5, 5))) // NCHW\nprint(\"X1:\", workspace.FetchBlob(\"X1\"))\nprint(\"X2:\", workspace.FetchBlob(\"X2\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\nprint(\"split_info:\", workspace.FetchBlob(\"split_info\"))\n\n```\n\n**Result**\n\n```\n\nX1: [[[[1 8 3 9 0]\n   [6 4 6 5 6]\n   [3 9 1 9 9]\n   [5 1 0 7 7]\n   [9 4 0 0 9]]]]\nX2: [[[[7 0 2 6 1]\n   [3 9 4 0 3]\n   [5 3 8 9 4]\n   [3 4 2 1 0]\n   [0 8 8 8 1]]]]\nY: [[[[[1 8 3 9 0]\n    [7 0 2 6 1]]\n\n   [[6 4 6 5 6]\n    [3 9 4 0 3]]\n\n   [[3 9 1 9 9]\n    [5 3 8 9 4]]\n\n   [[5 1 0 7 7]\n    [3 4 2 1 0]]\n\n   [[9 4 0 0 9]\n    [0 8 8 8 1]]]]]\nsplit_info: [1 1]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "name": "inputs",
          "option": "variadic"
        },
        {
          "description": "*(type: Tensor`<float>`)* List of input tensors.",
          "name": "X1, X2, ..."
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Concatenated tensor.",
          "name": "concat_result"
        },
        {
          "description": "*(type: Tensor`<int>`)* The dimensions of the inputs.",
          "name": "split_info"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GenerateProposals",
    "schema": {
      "attributes": [
        {
          "description": "(float) spatial scale",
          "name": "spatial_scale",
          "option": "optional"
        },
        {
          "description": "(int) RPN_PRE_NMS_TOP_N",
          "name": "pre_nms_topN",
          "option": "optional"
        },
        {
          "description": "(int) RPN_POST_NMS_TOP_N",
          "name": "post_nms_topN",
          "option": "optional"
        },
        {
          "description": "(float) RPN_NMS_THRESH",
          "name": "nms_thresh",
          "option": "optional"
        },
        {
          "description": "(float) RPN_MIN_SIZE",
          "name": "min_size",
          "option": "optional"
        },
        {
          "description": "bool (default false), Correct bounding box transform coordates, see bbox_transform() in boxes.py Set to true to match the detectron code, set to false for backward compatibility",
          "name": "correct_transform_coords",
          "option": "optional"
        },
        {
          "description": "bool (default true). If set, for rotated boxes, angle is normalized to be within [angle_bound_lo, angle_bound_hi].",
          "name": "angle_bound_on",
          "option": "optional"
        },
        {
          "description": "int (default -90 degrees). If set, for rotated boxes, angle is normalized to be within [angle_bound_lo, angle_bound_hi].",
          "name": "angle_bound_lo",
          "option": "optional"
        },
        {
          "description": "int (default 90 degrees). If set, for rotated boxes, angle is normalized to be within [angle_bound_lo, angle_bound_hi].",
          "name": "angle_bound_hi",
          "option": "optional"
        },
        {
          "description": "float (default 1.0 degrees). For RRPN, clip almost horizontal boxes within this threshold of tolerance for backward compatibility. Set to negative value for no clipping.",
          "name": "clip_angle_thresh",
          "option": "optional"
        }
      ],
      "description": "\nGenerate bounding box proposals for Faster RCNN. The propoasls are generated for\na list of images based on image score 'score', bounding box regression result\n'deltas' as well as predefined bounding box shapes 'anchors'. Greedy\nnon-maximum suppression is applied to generate the final bounding boxes.\n",
      "inputs": [
        {
          "description": "Scores from conv layer, size (img_count, A, H, W)",
          "name": "scores"
        },
        {
          "description": "Bounding box deltas from conv layer, size (img_count, 4 * A, H, W)",
          "name": "bbox_deltas"
        },
        {
          "description": "Image info, size (img_count, 3), format (height, width, scale)",
          "name": "im_info"
        },
        {
          "description": "Bounding box anchors, size (A, 4)",
          "name": "anchors"
        }
      ],
      "outputs": [
        {
          "description": "Proposals, size (n x 5), format (image_index, x1, y1, x2, y2)",
          "name": "rois"
        },
        {
          "description": "scores of proposals, size (n)",
          "name": "rois_probs"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RoIAlign",
    "schema": {
      "attributes": [
        {
          "description": "(float) default 1.0; Spatial scale of the input feature map X relative to the input image. E.g., 0.0625 if X has a stride of 16 w.r.t. the input image.",
          "name": "spatial_scale",
          "option": "optional"
        },
        {
          "description": "(int) default 1; Pooled output Y's height.",
          "name": "pooled_h",
          "option": "optional"
        },
        {
          "description": "(int) default 1; Pooled output Y's width.",
          "name": "pooled_w",
          "option": "optional"
        },
        {
          "description": "(int) default -1; number of sampling points in the interpolation grid used to compute the output value of each pooled output bin. If > 0, then exactly sampling_ratio x sampling_ratio grid points are used. If <= 0, then an adaptive number of grid points are used (computed as ceil(roi_width / pooled_w), and likewise for height).",
          "name": "sampling_ratio",
          "option": "optional"
        }
      ],
      "description": "\nRegion of Interest (RoI) align operation as used in Mask R-CNN.\n",
      "inputs": [
        {
          "description": "4D feature map input of shape (N, C, H, W).",
          "name": "X"
        },
        {
          "description": "2D input of shape (R, 4 or 5) specifying R RoIs representing: batch index in [0, N - 1], x1, y1, x2, y2. The RoI coordinates are in the coordinate system of the input image. For inputs corresponding to a single image, batch index can be excluded to have just 4 columns.",
          "name": "RoIs"
        }
      ],
      "outputs": [
        {
          "description": "4D output of shape (R, C, pooled_h, pooled_w). The r-th batch element is a pooled feature map cooresponding to the r-th RoI.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BBoxTransform",
    "schema": {
      "attributes": [
        {
          "description": "vector<float> weights [wx, wy, ww, wh] for the deltas",
          "name": "weights",
          "option": "optional"
        },
        {
          "description": "bool (default true), transform the boxes to the scaled image space after applying the bbox deltas.Set to false to match the detectron code, set to true for keypoint models and for backward compatibility",
          "name": "apply_scale",
          "option": "optional"
        },
        {
          "description": "bool (default false), Correct bounding box transform coordates, see bbox_transform() in boxes.py Set to true to match the detectron code, set to false for backward compatibility",
          "name": "correct_transform_coords",
          "option": "optional"
        },
        {
          "description": "bool (default false). If true, then boxes (rois and deltas) include angle info to handle rotation. The format will be [ctr_x, ctr_y, width, height, angle (in degrees)].",
          "name": "rotated",
          "option": "optional"
        },
        {
          "description": "bool (default true). If set, for rotated boxes, angle is normalized to be within [angle_bound_lo, angle_bound_hi].",
          "name": "angle_bound_on",
          "option": "optional"
        },
        {
          "description": "int (default -90 degrees). If set, for rotated boxes, angle is normalized to be within [angle_bound_lo, angle_bound_hi].",
          "name": "angle_bound_lo",
          "option": "optional"
        },
        {
          "description": "int (default 90 degrees). If set, for rotated boxes, angle is normalized to be within [angle_bound_lo, angle_bound_hi].",
          "name": "angle_bound_hi",
          "option": "optional"
        },
        {
          "description": "float (default 1.0 degrees). For RRPN, clip almost horizontal boxes within this threshold of tolerance for backward compatibility. Set to negative value for no clipping.",
          "name": "clip_angle_thresh",
          "option": "optional"
        }
      ],
      "description": "\nTransform proposal bounding boxes to target bounding box using bounding box\n    regression deltas.\n",
      "inputs": [
        {
          "description": "Bounding box proposals in pixel coordinates, Size (M, 4), format [x1, y1, x2, y2], orSize (M, 5), format [batch_index, x1, y1, x2, y2]. If proposals from multiple images in a batch are present, they should be grouped sequentially and in incremental order.For rotated boxes, this would have an additional angle (in degrees) in the format [<optionaal_batch_id>, ctr_x, ctr_y, w, h, angle].",
          "name": "rois"
        },
        {
          "description": "bounding box translations and scales,size (M, 4*K), format [dx, dy, dw, dh], K = # classes. For rotated boxes, size (M, 5*K, format [dx, dy, dw, dh, da].",
          "name": "deltas"
        },
        {
          "description": "Image dimensions, size (batch_size, 3), format [img_height, img_width, img_scale]",
          "name": "im_info"
        }
      ],
      "outputs": [
        {
          "description": "Pixel coordinates of the transformed bounding boxes,Size (M, 4*K), format [x1, y1, x2, y2]. For rotated boxes, size (M, 5*K), format [ctr_x, ctr_y, w, h, angle].",
          "name": "box_out"
        },
        {
          "description": "Tensor of shape (batch_size) with each element denoting the number of RoIs belonging to the corresponding image in batch",
          "name": "roi_batch_splits"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BoxWithNMSLimit",
    "schema": {
      "attributes": [
        {
          "description": "(float) TEST.SCORE_THRESH",
          "name": "score_thresh",
          "option": "optional"
        },
        {
          "description": "(float) TEST.NMS",
          "name": "nms",
          "option": "optional"
        },
        {
          "description": "(int) TEST.DEECTIONS_PER_IM",
          "name": "detections_per_im",
          "option": "optional"
        },
        {
          "description": "(bool) TEST.SOFT_NMS.ENABLED",
          "name": "soft_nms_enabled",
          "option": "optional"
        },
        {
          "description": "(string) TEST.SOFT_NMS.METHOD",
          "name": "soft_nms_method",
          "option": "optional"
        },
        {
          "description": "(float) TEST.SOFT_NMS.SIGMA",
          "name": "soft_nms_sigma",
          "option": "optional"
        },
        {
          "description": "(float) Lower bound on updated scores to discard boxes",
          "name": "soft_nms_min_score_thres",
          "option": "optional"
        },
        {
          "description": "bool (default false). If true, then boxes (rois and deltas) include angle info to handle rotation. The format will be [ctr_x, ctr_y, width, height, angle (in degrees)].",
          "name": "rotated",
          "option": "optional"
        }
      ],
      "description": "\nApply NMS to each class (except background) and limit the number of\nreturned boxes.\n",
      "inputs": [
        {
          "description": "Scores, size (count, num_classes)",
          "name": "scores"
        },
        {
          "description": "Bounding box for each class, size (count, num_classes * 4). For rotated boxes, this would have an additional angle (in degrees) in the format [<optionaal_batch_id>, ctr_x, ctr_y, w, h, angle]. Size: (count, num_classes * 5).",
          "name": "boxes"
        },
        {
          "description": "Tensor of shape (batch_size) with each element denoting the number of RoIs/boxes belonging to the corresponding image in batch. Sum should add up to total count of scores/boxes.",
          "name": "batch_splits"
        }
      ],
      "outputs": [
        {
          "description": "Filtered scores, size (n)",
          "name": "scores"
        },
        {
          "description": "Filtered boxes, size (n, 4). For rotated boxes, size (n, 5), format [ctr_x, ctr_y, w, h, angle].",
          "name": "boxes"
        },
        {
          "description": "Class id for each filtered score/box, size (n)",
          "name": "classes"
        },
        {
          "description": "Output batch splits for scores/boxes after applying NMS",
          "name": "batch_splits"
        },
        {
          "description": "Optional filtered indices, size (n)",
          "name": "keeps"
        },
        {
          "description": "Optional number of filtered indices per class, size (num_classes)",
          "name": "keeps_size"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ONNXWhile",
    "schema": {
      "attributes": [
        {
          "description": "Net executed on each iteration",
          "name": "body",
          "option": "optional"
        },
        {
          "description": "Whether to use the trip count input",
          "name": "has_trip_count",
          "option": "optional"
        },
        {
          "description": "Whether to use the condition input",
          "name": "has_cond",
          "option": "optional"
        },
        {
          "description": "Whether to save the scopes across iterations, as in for backprop",
          "name": "save_scopes",
          "option": "optional"
        },
        {
          "description": "Do not create new scopes. Use this only if you're certain there will be no name collision, for example if you're converting from a fully-SSA IR",
          "name": "disable_scopes",
          "option": "optional"
        }
      ],
      "description": "\n*** EXPERIMENTAL. This operator is a work-in-progress. No assumption should be\nmade about the stability or correctness of this op. ***\n\nGeneric Looping construct confirming to the ONNX Loop operator spec. This loop\nhas multiple termination conditions:\n\n1. Trip count. Iteration count specified at runtime. Set by specifying the\n    input M. Optional. Set to empty string to omit. Note that a static trip\n    count (specified at graph construction time) can be specified by passing\n    in a constant node for input M.\n2. Loop termination condition. This is an input to the op that determines\n    whether to run the first interation and also a loop-carried dependency for\n    the body graph. The body graph must yield a value for the condition\n    variable, whether this input is provided or not.\n\nThis table summarizes the operating modes of this operator with equivalent\nC-style code:\n\nOperator inputs defined as (max_trip_count, condition_var). Omitted optional\ninputs are represented as empty string. Concretely, in this caffe2 op an input\nis marked as omitted by setting its 'has_{name}' argument to False.\n\n    input (\"\", \"\"):\n        for (int i=0; ; ++i) {\n          cond = ... // Note this value is ignored, but is required in the body\n        }\n\n    input (\"\", cond) // Note this is analogous to a while loop\n        bool cond = ...;\n        for (int i=0; cond; ++i) {\n          cond = ...;\n        }\n\n    input (\"\", 1) // Note this is analogous to a do-while loop\n        bool cond = true\n        for (int i=0; cond; ++i) {\n          cond = ...;\n        }\n\n    input (trip_count, \"\") // Note this is analogous to a for loop\n        int trip_count = ...\n        for (int i=0; i < trip_count; ++i) {\n          cond = ...; // ignored\n        }\n\n    input (trip_count, cond)\n        int trip_count = ...;\n        bool cond = ...;\n        for (int i=0; i < trip_count && cond; ++i) {\n          cond = ...;\n        }\n    ",
      "inputs": [
        {
          "description": "Number of iterations to go out to. Used if the flag has_trip_count is True.",
          "name": "max_trip_count"
        },
        {
          "name": "condition"
        },
        {
          "name": "initial",
          "option": "variadic"
        },
        {
          "description": "Dynamic condition value for the first iteration. For all subsequent iterations, the condition from the body graph is used. This input is used if the flag has_cond is true.",
          "name": "first_iter_condition"
        }
      ],
      "outputs": [
        {
          "name": "final_and_scan_outputs",
          "option": "variadic"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8Quantize",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "name": "order"
        },
        {
          "default": 0,
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "default": 0,
          "description": "Output tensor quantization offset"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": null,
      "inputs": [
        {
          "description": "FP32 Tensor X.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Int8 Tensor qX representing X with linear quantization.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8Conv",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "name": "order"
        },
        {
          "default": 0,
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "default": 0,
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        },
        {
          "default": 0,
          "name": "pad"
        },
        {
          "default": 1,
          "name": "stride"
        }
      ],
      "category": "Layer",
      "description": "\nThe convolution operator consumes an input vector, a filter blob\nand a bias blob and computes the output. \n[Only NHWC order is supported now]Note that other parameters, such as the stride and\nkernel size, or the pads' sizes in each direction are not necessary for input\nbecause they are provided by the ConvPoolOpBase operator. Various dimension\nchecks are done implicitly, and the sizes are specified in the Input docs for\nthis operator. As is expected, the filter is convolved with a subset of the\nimage and the bias is added; this is done throughout the image data and the\noutput is computed. As a side note on the implementation layout:\nconv_op_impl.h is the templated implementation of the conv_op.h file, which is\nwhy they are separate files.\n",
      "inputs": [
        {
          "description": "Input data blob from previous layer; has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and width. Note that this is for the NCHW usage. On the other hand, the NHWC Op has a different set of dimension constraints. ",
          "name": "X"
        },
        {
          "description": "The filter blob that will be used in the convolutions; has size (M x C x kH x kW), where C is the number of channels, and kH and kW are the height and width of the kernel.",
          "name": "filter"
        },
        {
          "description": "The 1D bias blob that is added through the convolution; has size (M).",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob that contains the result of the convolution. The output dimensions are functions of the kernel size, stride size, and pad lengths.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8FC",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "name": "order"
        },
        {
          "default": 0,
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "default": 0,
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "category": "Layer",
      "description": "\nComputes the result of passing an input vector X into a fully\nconnected layer with 2D weight matrix W and 1D bias vector b. That is,\nthe layer computes Y = X * W^T + b, where X has size (M x K),\nW has size (N x K), b has size (N), and Y has size (M x N),\nwhere M is often the batch size.\n\n\nNOTE: X does not need to explicitly be a 2D vector; rather, it will be\ncoerced into one. For an arbitrary n-dimensional tensor\nX \\in [a_0, a_1 * ... * a_{n-1}]. Only this case is supported!\nLastly, even though b is a 1D vector of size N, it is copied/resized to\nbe size (M x N) implicitly and added to each vector in the batch.\nEach of these dimensions must be matched correctly, or else the operator\nwill throw errors.\n",
      "inputs": [
        {
          "description": "input tensor that's coerced into a 2D matrix of size (MxK) as described above",
          "name": "X"
        },
        {
          "description": "A tensor that is coerced into a 2D blob of size (KxN) containing fully connected weight matrix",
          "name": "W"
        },
        {
          "description": "1D blob containing bias vector",
          "name": "b"
        }
      ],
      "outputs": [
        {
          "description": "2D output tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8AveragePool",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "name": "order"
        },
        {
          "default": 0,
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "default": 0,
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "category": "Pool",
      "description": "AveragePool \nconsumes an input blob X and applies average pooling across the\nthe blob according to kernel sizes, stride sizes, and pad lengths defined by the\nConvPoolOpBase operator. Average pooling consisting of averaging all values of a\nsubset of the input tensor according to the kernel size and downsampling the\ndata into the output blob Y for further processing.\n",
      "inputs": [
        {
          "description": "Input data tensor from the previous operator; dimensions depend on whether the NCHW or NHWC operators are being used. For example, in the former, the input has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. The corresponding permutation of dimensions is used in the latter case.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output data tensor from average pooling across the input tensor. Dimensions will vary based on various kernel, stride, and pad sizes.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8Sum",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "name": "order"
        },
        {
          "default": 0,
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "default": 0,
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Int8Softmax",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "name": "order"
        },
        {
          "default": 0,
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "default": 0,
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        },
        {
          "description": "(int) default to 1; describes the axis of the inputs when coerced to 2D; defaults to one because the 0th axis most likely describes the batch_size",
          "name": "axis",
          "option": "optional"
        }
      ],
      "category": "Activation",
      "description": "\nThe operator computes the softmax normalized values for each layer in the batch\n of the given input. The input is a 2-D tensor (Tensor<float>) of size\n(batch_size x input_feature_dimensions). The output tensor has the same shape\nand contains the softmax normalized values of the corresponding input.\n\nX does not need to explicitly be a 2D vector; rather, it will be\ncoerced into one. For an arbitrary n-dimensional tensor\nX \\in [a_0, a_1, ..., a_{k-1}, a_k, ..., a_{n-1}] and k is\nthe axis provided, then X will be coerced into a 2-dimensional tensor with\ndimensions [a_0 * ... * a_{k-1}, a_k * ... * a_{n-1}]. For the default\ncase where axis=1, this means the X tensor will be coerced into a 2D tensor\nof dimensions [a_0, a_1 * ... * a_{n-1}], where a_0 is often the batch size.\nIn this situation, we must have a_0 = N and a_1 * ... * a_{n-1} = D.\nEach of these dimensions must be matched correctly, or else the operator\nwill throw errors.\n",
      "inputs": [
        {
          "description": "The input tensor that's coerced into a 2D matrix of size (NxD) as described above.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The softmax normalized output values with the same shape as input tensor.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8Relu",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "name": "order"
        },
        {
          "default": 0,
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "default": 0,
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "category": "Activation",
      "description": "\nRelu takes one input data (Tensor<T>) and produces one output data\n(Tensor<T>) where the rectified linear function, y = max(0, x), is applied to\nthe tensor elementwise.\n",
      "inputs": [
        {
          "description": "1D input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "1D input tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AffineChannel",
    "schema": {
      "category": "Normalization",
      "description": "\nApplies a separate affine transformation to each channel of the input. Useful\nfor replacing spatial batch norm with its equivalent fixed transformation.\n",
      "inputs": [
        {
          "description": "Feature map input with order NCHW or NHWC.",
          "name": "X"
        },
        {
          "description": "1D input of shape (C); the c-th element is the scale factor of the affine transformation for the c-th channel of the input.",
          "name": "scale"
        },
        {
          "description": "1D input of shape (C); the c-th element is the bias of the affine transformation for the c-th channel of the input.",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output with the same order of Input.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LearningRateAdaption",
    "schema": {
      "attributes": [
        {
          "description": "the learning rate for performing gradient descent on learning rate lr",
          "name": "lr_alpha",
          "option": "optional"
        },
        {
          "description": "whether to apply normalized lr adaption or not",
          "name": "normalized_lr_adaption",
          "option": "optional"
        }
      ],
      "description": "\n      Learning Rate Adaption is an operation that perform one iteration of\n      gradient descent based on learning rate:\n        lr(k) = lr(k-1) - lr_alpha * df(k-1)/dlr,\n      where df(k-1)/dlr is the gradient of objective function f on lr, and\n      lr_alpha is a learning rate hyperparameter. It can be prove that\n      df(k-1)/dlr equals INNERPRODUCT(grad(k-1), -grad(k-2)), where grad(k-1) is\n      the grad of f(k-1) on parameters. When the argument\n      \"normalized_lr_adaption\" is false, we simply perform the\n      following update:\n      lr(k) = lr(k-1) - lr_alpha * INNERPRODUCT(grad(k-1), grad(k-2)).\n      If we set \"normalized_lr_adaption\" to be true, we do not directly apply\n      INNERPRODUCT(grad(k-1), -grad(k-2)) as the grad. Instead, we perform the\n      following update:\n      lr(k) = lr(k-1) + lr_alpha * cosineSimilarity(grad(k-1), grad(k-2)).\n",
      "inputs": [
        {
          "description": "Learning rate",
          "name": "lr"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "The effective grad",
          "name": "effgrad"
        }
      ],
      "outputs": [
        {
          "description": "Updated learning rate",
          "name": "output_lr"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CoshGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "IndexSize",
    "schema": {
      "description": "\nReturns the number of entries currently present in the index.\n",
      "inputs": [
        {
          "description": "Pointer to an Index instance.",
          "name": "handle"
        }
      ],
      "outputs": [
        {
          "description": "Scalar int64 tensor with number of entries.",
          "name": "items"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LpPool",
    "schema": {
      "attributes": [
        {
          "description": "(*float*): type of $L_p$ norm to use (default=2.0)",
          "name": "p",
          "option": "optional"
        },
        {
          "description": "(*int*): the size of the window to take a max over",
          "name": "kernel",
          "option": "optional"
        },
        {
          "description": "(*int*): the stride of the window",
          "name": "stride",
          "option": "optional"
        },
        {
          "description": "(*int*): implicit zero padding to be added on both sides",
          "name": "pad",
          "option": "optional"
        },
        {
          "description": "(*int*): parameter that controls the stride of elements in the window",
          "name": "dilation",
          "option": "optional"
        },
        {
          "description": "(*string*): order of blob dimensions (default=\"NCHW\")",
          "name": "order",
          "option": "optional"
        }
      ],
      "description": "\n`LpPool` consumes an input blob and applies max pooling across the the blob according to kernel sizes, stride sizes, pad lengths and dilation. $L_p$ pooling consists of taking the $L_p$ norm of a subset of the input tensor according to the kernel size and downsampling the data into the output blob for further processing.\n\nPooling layers reduce the spatial dimensionality of the input blob. Each of the output blob's dimensions will reduce according to:\n\n$$dim_{out}=\\frac{dim_{in}-kernel+2*pad}{stride}+1$$\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/lp_pool_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LpPool\",\n    [\"X\"],\n    [\"Y\"],\n    kernel=2,\n    stride=2,\n    p=2.0\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(1, 1, 6, 6).astype(np.float32)) // NCHW\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[[[-1.1113514  -1.1173418  -0.1504435   0.1327146  -1.2221841  -0.5654315 ]\n   [-1.9209646  -0.04675794  0.8604731   1.2042469   0.28154245   0.38656202]\n   [-0.8772837  -0.03264008  0.26222762  0.28526652  0.321102    -2.5891325 ]\n   [-0.9248281   1.440776   -0.56832    -0.6017927   1.2262512   -2.1443934 ]\n   [ 0.5194415  -1.6858683   0.45221648  0.65029615 -0.8574544    0.8121054 ]\n   [ 0.25902653  0.4934758   0.49870652 -0.48134378 -0.9178449   -0.07626943]]]]\n\nY:\n [[[[2.4851248 1.49361   1.4290358]\n   [1.9240153 0.9139378 3.5928857]\n   [1.8500228 1.0525136 1.4976646]]]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): output tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsMeanFused8BitRowwise",
    "schema": {
      "description": "\nPerforms the same operation as SparseLengthsMean, but\noperating on 8-bit rowwise quantized matrices with fused storage\n(where each row stores quantized values, and then 4-byte scale and 4-byte bias).\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused8BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Transpose",
    "schema": {
      "attributes": [
        {
          "description": "Order to permute axes of input tensor. Reverses the dimensions by default.",
          "name": "axes",
          "option": "optional"
        }
      ],
      "description": "\nTranspose the input tensor by permuting the axes of the input according\nto the `axes` argument. Similar to numpy's\n[transpose](https://docs.scipy.org/doc/numpy/reference/generated/numpy.transpose.html)\nfunction.\n\nFor example, when axes=(1, 0, 2), given an input tensor of shape\n(1, 2, 3), the output shape will be (2, 1, 3).\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/transpose_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Transpose\",\n    [\"X\"],\n    [\"Y\"],\n    axes=(0,3,1,2)\n)\n\nx = np.random.rand(1,32,32,3)\nworkspace.FeedBlob(\"X\", x)\nprint(\"X.shape (NHWC order):\", workspace.FetchBlob(\"X\").shape)\nworkspace.RunOperatorOnce(op)\nprint(\"Y.shape (NCHW order):\", workspace.FetchBlob(\"Y\").shape)\n```\n\n**Result**\n\n```\nX.shape (NHWC order): (1, 32, 32, 3)\nY.shape (NCHW order): (1, 3, 32, 32)\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor)* Transposed output.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Accuracy",
    "schema": {
      "attributes": [
        {
          "description": "Count as correct by comparing the true label to the top k scoring classes (default 1: only compare to the top scoring class i.e. argmax)",
          "name": "top_k",
          "option": "optional"
        }
      ],
      "description": "\nAccuracy takes two inputs- predictions and labels, and returns a float\naccuracy value for the batch. Predictions are expected in the form of 2-D tensor\ncontaining a batch of scores for various classes, and labels are expected in the\n form of 1-D tensor containing true label indices of samples in the batch. If\nthe score for the label index in the predictions is the highest among all\nclasses, it is considered a correct prediction.\n",
      "inputs": [
        {
          "description": "2-D tensor (Tensor<float>) of size (num_batches x num_classes) containing scores",
          "name": "predictions"
        },
        {
          "description": "1-D tensor (Tensor<float>) of size (num_batches) having the indices of true labels",
          "name": "labels"
        }
      ],
      "outputs": [
        {
          "description": "1-D tensor (Tensor<float>) of size 1 containing accuracy",
          "name": "accuracy"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "TimerEnd",
    "schema": {
      "description": "\nStop a timer started with **TimerBegin**. Publishes a CAFFE_EVENT.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/stats_ops.cc\n\n    ",
      "inputs": [
        {
          "description": "(*Tensor`<ptr>`*): pointer to a timer object; obtained from **TimerBegin** op",
          "name": "timer"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsRangeFill",
    "schema": {
      "description": "\nThe *LengthsRangeFill* op takes a single input *lengths* and outputs a single tensor *range_sequence*. For each element of *lengths*, the op appends the range(0,lengths) vector to the end of *range_sequence*. For example, if input=[2,4,1], the output would be [0,1,0,1,2,3,0].\n\nGithub Links:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/filler_op.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/filler_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LengthsRangeFill\",\n    [\"lengths\"],\n    [\"range_sequence\"],\n)\n\nworkspace.FeedBlob(\"lengths\", np.array([2,4,1]).astype(np.int32))\nprint(\"lengths:\\n\", workspace.FetchBlob(\"lengths\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"range_sequence: \\n\", workspace.FetchBlob(\"range_sequence\"))\n\n```\n\n**Result**\n\n```\n\nlengths:\n [2 4 1]\nrange_sequence:\n [0 1 0 1 2 3 0]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "1D tensor of int32 or int64 segment lengths.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "1D tensor whose size is the sum of *lengths*",
          "name": "range_sequence"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AccumulateHistogram",
    "schema": {
      "attributes": [
        {
          "description": "the lower bound value",
          "name": "lower_bound",
          "option": "optional"
        },
        {
          "description": "the upper bound value",
          "name": "upper_bound",
          "option": "optional"
        },
        {
          "description": "number of buckets to use in [lower_bound, upper_bound)",
          "name": "num_buckets",
          "option": "optional"
        }
      ],
      "description": "\nThis operator calculate thes histogram of values in input tensor.\nThere're 2 outputs, one for histogram of current input tensor, and another\nfor histogram of the all input tensors accumulated through history.\nThe output would contain num_buckets + 2 values. index[1 ... num_buckets]\nfor values in [lower_bound, upper_bound) interval. And the rest 2 for values\nsmaller than lower_bound or greater than upper_bound respectively.\n",
      "inputs": [
        {
          "description": "Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output histogram of the current tensor.",
          "name": "CurHist"
        },
        {
          "description": "Accumulated histogram of the history tensor.",
          "name": "AccHist"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8ConvRelu",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "\nThe convolution operator consumes an input vector, a filter blob\nand a bias blob and computes the output. \n[Only NHWC order is supported now]Note that other parameters, such as the stride and\nkernel size, or the pads' sizes in each direction are not necessary for input\nbecause they are provided by the ConvPoolOpBase operator. Various dimension\nchecks are done implicitly, and the sizes are specified in the Input docs for\nthis operator. As is expected, the filter is convolved with a subset of the\nimage and the bias is added; this is done throughout the image data and the\noutput is computed. As a side note on the implementation layout:\nconv_op_impl.h is the templated implementation of the conv_op.h file, which is\nwhy they are separate files.\n",
      "inputs": [
        {
          "description": "Input data blob from previous layer; has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and width. Note that this is for the NCHW usage. On the other hand, the NHWC Op has a different set of dimension constraints. ",
          "name": "X"
        },
        {
          "description": "The filter blob that will be used in the convolutions; has size (M x C x kH x kW), where C is the number of channels, and kH and kW are the height and width of the kernel.",
          "name": "filter"
        },
        {
          "description": "The 1D bias blob that is added through the convolution; has size (M).",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob that contains the result of the convolution. The output dimensions are functions of the kernel size, stride size, and pad lengths. Output will go through rectified linear function, where y = max(0, x).",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsSum8BitsRowwise",
    "schema": {
      "description": "\nVariation of SparseLengthsSum operator, where DATA is\nstored using 8bits. DATA was quantized with 8Bit row-wise\nquantization (see doc to FloatToRowwiseQuantized8Bits operator). To\nrestore DATA from 8Bit, we use additional input that stores scales\nand biases.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToRowwiseQuantized8Bits",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Matrix of floats, each row r_i of which stores a pair s_i, b_i -- scale and bias for i-th row",
          "name": "scale_bias"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseUnsortedSegmentMean",
    "schema": {
      "description": "\nPulls in slices of the input tensor, groups them into segments and applies\n'Mean' to each segment. Segments ids can appear in arbitrary order (unlike in\nSparseSortedSegmentMean).\n\nThis op is basically Gather and UnsortedSegmentMean fused together.\n\nINDICES should contain integers in range 0..N-1 where N is the first dimension\nof DATA. INDICES represent which slices of DATA need to be pulled in.\n\nSEGMENT_IDS is a vector that maps each referenced slice of the DATA to a\nparticular group (segment). Values belonging to the same segment are aggregated\ntogether. SEGMENT_IDS should have the same dimension as INDICES.\n\nIf `num_segments` argument is passed it would be used as a first dimension for\nthe output. Otherwise, it'd be dynamically calculated from as the max value of\nSEGMENT_IDS plus one. Other output dimensions are inherited from the input\ntensor.\n\nMean computes the element-wise mean of the input slices. Operation doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Integer vector with the same length as INDICES that maps each slice of DATA referenced by INDICES to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of equal to the number of segments.",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Load",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "If set to non-zero, save the db directly to the path specified by the `db` arg. If not set (default), prepend the path of the current root folder of the workspace to the path specified by the `db` arg.",
          "name": "absolute_path",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": "",
          "description": "Blobs will be prefixed with this when loading. Useful for avoiding collisions with blobs existing in the workspace. The output blob names specified to this op should include this prefix.",
          "name": "add_prefix",
          "option": "optional",
          "type": "string"
        },
        {
          "default": "",
          "description": "Characters in the provided blob names that match `strip_prefix` will be removed prior to saving. Also, characters that precede `strip_prefix` will be removed. Useful for removing device scope from blob names.",
          "name": "strip_prefix",
          "option": "optional",
          "type": "string"
        },
        {
          "description": "The output path of the db. See the `absolute_path` arg details for options regarding the current root folder of the workspace.",
          "name": "db",
          "option": "optional",
          "type": "string"
        },
        {
          "description": "List of paths to dbs to load blobs from. See the `absolute_path` arg details for options regarding the current root folder of the workspace.",
          "name": "dbs",
          "option": "optional",
          "type": "string[]"
        },
        {
          "description": "(type: string)* Type of db to save (options: \"lmdb\", \"leveldb\", \"minidb\").",
          "name": "db_type",
          "option": "optional"
        },
        {
          "default": 0,
          "description": "If nonzero, the blobs are loaded into the device that is specified in the serialized `BlobProto`. Otherwise, the device will be set as the one that the `Load` operator is being run under.",
          "name": "keep_device",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": 0,
          "description": "If nonzero, will load all blobs pointed to by the db to the workspace overwriting/creating blobs as needed.",
          "name": "load_all",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": false,
          "description": "If True, will allow not loading all the output blobs specified in the outputs.",
          "name": "allow_incomplete",
          "option": "optional",
          "type": "boolean"
        },
        {
          "description": "If set, used instead of output blob names to specify which blobs in the db shall be loaded. Must be the same length as number of output blobs.",
          "name": "source_blob_names",
          "option": "optional",
          "type": "string[]"
        }
      ],
      "description": "\nThe Load operator loads a set of serialized blobs from a db or multiple dbs. It\ntakes $[0, \\infty)$ number of inputs and $[0, \\infty)$ number of outputs, using\nthe db keys to match the db entries with the outputs.\n\nIf at least one input is passed, then it is assumed that that input blobs are a\nset of DBReaders to load from. Otherwise the `db` or `dbs` argument is used to load\nblobs from one single db or multiple dbs respectively. `db_type` argument is used\nto specify the type of the input db/dbs.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/load_save_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Load\",\n    [],\n    [\"X\", \"Y\"],\n    db=\"test_db\",\n    db_type=\"lmdb\"\n)\n\nworkspace.RunOperatorOnce(op)\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: List(DBReader))* [OPTIONAL] List of DBReaders to load from. Can use this instead of the `db`/`dbs` args.",
          "name": "X, Y, ..."
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Exp",
    "schema": {
      "description": "\nCalculates the exponential of the given input tensor ($exp(x)$), element-wise. This\noperation can be done in an in-place fashion too, by providing the same input\nand output blobs.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/exp_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Exp\",\n    [\"X\"],\n    [\"X\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.rand(3,3)).astype(np.float32))\nprint(\"X before running op:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"X after running op:\", workspace.FetchBlob(\"X\"))\n\n```\n\n**Result**\n\n```\n\nX before running op:\n[[0.5821691  0.07719802 0.50159824]\n [0.40952456 0.36788362 0.84887683]\n [0.02472685 0.65730894 0.9066397 ]]\nX after running op:\n[[1.7899168 1.080256  1.6513585]\n [1.5061016 1.4446739 2.3370204]\n [1.0250351 1.9295927 2.4759884]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* The exponential of the input tensor computed element-wise.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ConvTransposeGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LayerNormGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SinhGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "FlattenToVec",
    "schema": {
      "description": "\n\nThe *FlattenToVec* op flattens the input tensor into a 1-D vector. The op accepts a single input tensor and returns a single output tensor.\n\nGithub Links:\n\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/utility_ops.cc\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/utility_ops.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"FlattenToVec\",\n    [\"input\"],\n    [\"output\"],\n)\n\nworkspace.FeedBlob(\"input\", np.array([[1,2,3],[4,5,6],[7,8,9],[10,11,12]]).astype(np.float32))\nprint(\"input:\\n\", workspace.FetchBlob(\"input\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"output: \\n\", workspace.FetchBlob(\"output\"))\n\n```\n\n**Result**\n\n```\n\ninput:\n [[ 1.  2.  3.]\n [ 4.  5.  6.]\n [ 7.  8.  9.]\n [10. 11. 12.]]\noutput:\n [ 1.  2.  3.  4.  5.  6.  7.  8.  9. 10. 11. 12.]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "A tensor of rank >= 1.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "A tensor of rank 1 (vector) with the contents of the input tensor.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Ftrl",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "UnsortedSegmentMean",
    "schema": {
      "attributes": [
        {
          "description": "Optional int argument specifying the number of output segments and thus the first dimension of the output",
          "name": "num_segments",
          "option": "optional"
        }
      ],
      "description": "\nApplies 'Mean' to each segment of input tensor. Segments ids can appear in\narbitrary order (unlike in SortedSegmentMean).\n\nSEGMENT_IDS is a vector that maps each of the first dimension slices of the\nDATA to a particular group (segment). Values belonging to the same segment are\naggregated together.\n\nIf `num_segments` argument is passed it would be used as a first dimension for\nthe output. Otherwise, it'd be dynamically calculated from as the max value of\nSEGMENT_IDS plus one. Other output dimensions are inherited from the input\ntensor.\n\nMean computes the element-wise mean of the input slices. Operation doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Integer vector with the same length as the first dimension of DATA that maps each slice of DATA to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of equal to the number of segments.",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Sinh",
    "schema": {
      "description": "\nCalculates the hyperbolic sine of the given input tensor, element-wise.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/sinh_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Sinh\",\n    [\"X\"],\n    [\"Y\"]\n)\n\nworkspace.FeedBlob(\"X\", np.random.rand(5).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX: [0.98907769 0.52907848 0.03216429 0.94983935 0.47881418]\nY: [1.15841695 0.5541099  0.03216984 1.09924557 0.49732079]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The hyperbolic sine values of the input tensor, computed element-wise",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CloseRebatchingQueue",
    "schema": {
      "description": "\nCloses the Queue.\n",
      "inputs": [
        {
          "description": "object representing the queue",
          "name": "queue"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LpPoolGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "StumpFunc",
    "schema": {
      "description": "\nConverts each input element into either high_ or low_value\nbased on the given threshold.\n",
      "inputs": [
        {
          "description": "tensor of float",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "tensor of float",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BooleanMask",
    "schema": {
      "description": "\nGiven a 1D `data` tensor and a boolean `mask` tensor of the same shape, returns a `masked_data` tensor containing only the elements corresponding to positions where the `mask` is True, and a `masked_indices` tensor containing the indices of the True elements.\n\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/boolean_mask_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"BooleanMask\",\n    [\"data\", \"mask\"],\n    [\"masked_data\", \"masked_indices\"]\n)\n\nworkspace.FeedBlob(\"data\", np.array([1,2,3,4,5,6]))\nworkspace.FeedBlob(\"mask\", np.array([True,False,False,True,True,False]))\nprint(\"data:\", workspace.FetchBlob(\"data\"))\nprint(\"mask:\", workspace.FetchBlob(\"mask\"))\nworkspace.RunOperatorOnce(op)\nprint(\"masked_data:\", workspace.FetchBlob(\"masked_data\"))\nprint(\"masked_indices:\", workspace.FetchBlob(\"masked_indices\"))\n\n```\n\n**Result**\n\n```\n\ndata: [1 2 3 4 5 6]\nmask: [ True False False  True  True False]\nmasked_data: [1 4 5]\nmasked_indices: [0 3 4]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor*): 1D input tensor",
          "name": "data"
        },
        {
          "description": "(*Tensor`<bool>`*): tensor of bools which determines the input elements that will be left in the `masked_data` output tensor; same shape as `data`",
          "name": "mask"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor*): 1D tensor of same type as `data` input that contains the masked input tensor",
          "name": "masked_data"
        },
        {
          "description": "(*Tensor`<int>`*): 1D tensor of indices of the True elements in the `mask` tensor",
          "name": "masked_indices"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceFrontMean",
    "schema": {
      "attributes": [
        {
          "description": "(*int*): number of dimensions to reduce (default=1)",
          "name": "num_reduce_dims",
          "option": "optional"
        }
      ],
      "description": "\nReduces the input tensor along the last dimension of the by applying **mean**.\n\nCan reduce more than one of the \"first\" dimensions by setting `num_reduce_dim`.\n\nA second (optional) input, `lengths`, can be passed, which enforces that only a subset of the elements are considered in the mean operation.\n- If input tensor `X` has shape $(d_0, d_1, d_2, ..., d_n)$, `lengths` must have shape $(d_1 * d_2 * ... * d_{n})$.\n- The values of the `lengths` tensor determine how many of the values to consider for each vector in the $d_{0}$ dimension.\n\nFor example if $X = [[1,5,2,9],[4,1,8,2],[2,7,0,3]]$ and $lengths = [2,3,1,2]$, then $Y = [mean(1,4), mean(5,1,7), mean(2), mean(9,2)] = [2.5, 4.333, 2, 5.5]$\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduce_front_back_mean_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ReduceFrontMean\",\n    [\"X\"],\n    [\"Y\"],\n    num_reduce_dim=2\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(2,3,3)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[[5. 0. 9.]\n  [4. 1. 1.]\n  [9. 0. 8.]]\n\n [[2. 6. 7.]\n  [6. 2. 6.]\n  [0. 4. 5.]]]\nY: [4.3333335    2.1666667     6.]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor",
          "name": "X"
        },
        {
          "description": "(*Tensor`<int>`*): number of elements in each sample",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): reduced tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SigmoidCrossEntropyWithLogits",
    "schema": {
      "attributes": [
        {
          "description": "default is false; if enabled, will use the log d trick to avoid the vanishing\ngradients early on; see Goodfellow et. al (2014)",
          "name": "log_D_trick",
          "option": "optional"
        },
        {
          "description": "default is false; if enabled, the model will be allowed to train on an unjoined\ndataset, where some examples might be false negative and might appear\nin the dataset later as (true) positive example.",
          "name": "unjoined_lr_loss",
          "option": "optional"
        }
      ],
      "description": "\nGiven two matrices logits and targets, of same shape,\n(batch_size, num_classes), computes the sigmoid cross entropy between the two.\nReturns a tensor of shape (batch_size,) of losses for each example.\n",
      "inputs": [
        {
          "description": "matrix of logits for each example and class.",
          "name": "logits"
        },
        {
          "description": "matrix of targets, same shape as logits.",
          "name": "targets"
        }
      ],
      "outputs": [
        {
          "description": "Vector with the total xentropy for each example.",
          "name": "xentropy"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CosineEmbeddingCriterionGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ResizeLike",
    "schema": {
      "description": "\nProduces tensor containing data of first input and shape of second input.\n",
      "inputs": [
        {
          "description": "Tensor whose data will be copied into the output.",
          "name": "data"
        },
        {
          "description": "Tensor whose shape will be applied to output.",
          "name": "shape_tensor"
        }
      ],
      "outputs": [
        {
          "description": "Tensor with data of input 0 and shape of input 1.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HSoftmaxSearch",
    "schema": {
      "attributes": [
        {
          "description": "Serialized TreeProto string containing a tree including all intermidate nodes and leafs. All nodes must have names for correct outputs",
          "name": "tree",
          "option": "optional"
        },
        {
          "description": "beam used for pruning tree. The pruning algorithm is that only children, whose score is smaller than parent's score puls beam, will be propagated.",
          "name": "beam",
          "option": "optional"
        },
        {
          "description": "Number of nodes in outputs",
          "name": "topN",
          "option": "optional"
        }
      ],
      "description": "\nHSoftmaxSearch is an operator to generate the most possible paths given a\nwell-trained model and input vector. Greedy algorithm is used for pruning the\nsearch tree.\n",
      "inputs": [
        {
          "description": "Input data from previous layer",
          "name": "X"
        },
        {
          "description": "The matrix trained from Softmax Ops",
          "name": "W"
        },
        {
          "description": "The bias trained from Softmax Ops",
          "name": "b"
        }
      ],
      "outputs": [
        {
          "description": "The name of selected nodes and leafs. For nodes, it will be the name defined in the tree. For leafs, it will be the index of the word in the tree.",
          "name": "Y_names"
        },
        {
          "description": "The corresponding scores of Y_names",
          "name": "Y_scores"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HSoftmaxGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CloneCommonWorld",
    "schema": {
      "description": "\nClones existing common world.\n",
      "inputs": [
        {
          "description": "Existing common world to clone.",
          "name": "existing_comm_world"
        }
      ],
      "outputs": [
        {
          "description": "A common world for collective operations.",
          "name": "comm_world"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentRangeMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LCGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SubGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "PackedInt8BGRANHWCToNCHWCStylizerPreprocess",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ConcatBatchMatMulBatchGatherOp",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Gather",
    "schema": {
      "description": "\n\nThe *Gather* op accepts a *DATA* tensor of rank $r >= 1$ and *INDICES* tensor of rank $q$ as inputs. It then gathers entries of the outer-most dimension of *DATA*, indexed by *INDICES*, and concatenate them in an output tensor of rank $q + (r - 1)$.\n\nGithub Links:\n\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/gather_op.cc\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/gather_op.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Gather\",\n    [\"DATA\", \"INDICES\"],\n    [\"OUTPUT\"]\n)\ndata = np.array([[1., 1.2],[2.3, 3.4],[4.5, 5.7]])\nprint(\"DATA:\\n\",data)\n\ninds = np.array([[0, 1],[1, 2]])\nprint(\"INDICES:\\n\",inds)\n\n// Feed X into workspace\nworkspace.FeedBlob(\"DATA\", data.astype(np.float32))\nworkspace.FeedBlob(\"INDICES\", inds.astype(np.int32))\n\nworkspace.RunOperatorOnce(op)\nprint(\"OUTPUT:\\n\", workspace.FetchBlob(\"OUTPUT\"))\n\n```\n\n**Result**\n\n```\n\nDATA:\n [[1.  1.2]\n [2.3 3.4]\n [4.5 5.7]]\nINDICES:\n [[0 1]\n [1 2]]\nOUTPUT:\n [[[1.  1.2]\n  [2.3 3.4]]\n\n [[2.3 3.4]\n  [4.5 5.7]]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "Input data tensor of rank $r>=1$",
          "name": "DATA"
        },
        {
          "description": "Input indices tensor of rank $q$. This tensor must contain integers.",
          "name": "INDICES"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor of rank $q+(r-1)$",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "KeyValueToMap",
    "schema": {
      "description": "Convert key and value blob pairs into a map blob",
      "inputs": [
        {
          "description": "Blob reference to the key",
          "name": "key blob"
        },
        {
          "description": "Blob reference to the value",
          "name": "value blob"
        }
      ],
      "outputs": [
        {
          "description": "Blob reference to the map",
          "name": "map blob"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Unique",
    "schema": {
      "description": "\nDeduplicates input indices vector and optionally produces reverse remapping.\nThere's no guarantees on the ordering of the output indices.\n",
      "inputs": [
        {
          "description": "1D tensor of int32 or int64 indices.",
          "name": "indices"
        }
      ],
      "outputs": [
        {
          "description": "1D tensor of deduped entries.",
          "name": "unique_indices"
        },
        {
          "description": "(optional) mapping from `indices` to `unique_indices`. This has the same shape as `indices`. Its elements are the indices into `unique_indices` such that `Gather(['unique_indices', 'remapping'])` yields `indices`.",
          "name": "remapping"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ResizeNearestGradient",
    "schema": {
      "attributes": [
        {
          "description": "Scale along width dimension",
          "name": "width_scale",
          "option": "optional"
        },
        {
          "description": "Scale along height dimension",
          "name": "height_scale",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "AveragePut",
    "schema": {
      "attributes": [
        {
          "description": "(*str*): name of the stat. If not present, then uses name of input blob",
          "name": "name",
          "option": "optional"
        },
        {
          "description": "(*int64_t*): number to multiply input values by (used when inputting floats, as stats can only receive integers",
          "name": "magnitude_expand",
          "option": "optional"
        },
        {
          "description": "(*boolean*): whether or not to clamp inputs to the max inputs allowed",
          "name": "bound",
          "option": "optional"
        },
        {
          "description": "(*float*): Optionally provide a default value for receiving empty tensors",
          "name": "default_value",
          "option": "optional"
        }
      ],
      "description": "\n    Consume a value and pushes it to the global stat registry as an average.\n\n    Github Links:\n    - https://github.com/pytorch/pytorch/blob/master/caffe2/operators/stats_put_ops.cc\n\n        ",
      "inputs": [
        {
          "description": "(*Tensor`<number>`*): A scalar tensor, representing any numeric value",
          "name": "value"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SoftmaxGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "BatchBucketize",
    "schema": {
      "description": "\nBucketize the float_features into sparse features.\nThe float_features is a N * D tensor where N is the batch_size, and D is the feature_dim.\nThe indices is a 1D tensor containing the indices of the features that need to be bucketized.\nThe lengths is a 1D tensor that splits the following 'boundaries' argument.\nThe boundaries is a 1D tensor containing the border list for each feature.\n\nWith in each batch, `indices` should not have duplicate number,\nand the number of elements in `indices` should be less than or equal to `D`.\nEach element in `lengths` vector (lengths[`i`]) represents\nthe number of boundaries in the sub border list.\nThe sum of all elements in `lengths` must be equal to the size of  `boundaries`.\nIf lengths[0] = 2, the first sub border list is [0.5, 1.0], which separate the\nvalue to (-inf, 0.5], (0,5, 1.0], (1.0, inf). The bucketized feature will have\nthree possible values (i.e. 0, 1, 2).\n\n\nFor example, with input:\n\n  float_features = [[1.42, 2.07, 3.19, 0.55, 4.32],\n                    [4.57, 2.30, 0.84, 4.48, 3.09],\n                    [0.89, 0.26, 2.41, 0.47, 1.05],\n                    [0.03, 2.97, 2.43, 4.36, 3.11],\n                    [2.74, 5.77, 0.90, 2.63, 0.38]]\n  indices = [0, 1, 4]\n  lengths = [2, 3, 1]\n  boundaries =  [0.5, 1.0, 1.5, 2.5, 3.5, 2.5]\n\nThe output is:\n\n  output =[[2, 1, 1],\n           [2, 1, 1],\n           [1, 0, 0],\n           [0, 2, 1],\n           [2, 3, 0]]\n\nafter running this operator.\n",
      "inputs": [
        {
          "description": "2-D dense tensor, the second dimension must be greater or equal to the indices dimension",
          "name": "float_features"
        },
        {
          "description": "Flatten tensor, containing the indices of `float_features` to be bucketized. The datatype must be int32.",
          "name": "indices"
        },
        {
          "description": "Flatten tensor, the size must be equal to that of `indices`. The datatype must be int32.",
          "name": "lengths"
        },
        {
          "description": "Flatten tensor, dimension has to match the sum of lengths",
          "name": "boundaries"
        }
      ],
      "outputs": [
        {
          "description": "2-D dense tensor, with 1st dim = float_features.dim(0), 2nd dim = size(indices)in the arg list, the tensor is of the same data type as `feature`.",
          "name": "bucktized_feat"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CreateTensorVector",
    "schema": {
      "description": "Create a std::unique_ptr<std::vector<Tensor> >",
      "support_level": "default"
    }
  },
  {
    "name": "UnsortedSegmentSum",
    "schema": {
      "attributes": [
        {
          "description": "Optional int argument specifying the number of output segments and thus the first dimension of the output",
          "name": "num_segments",
          "option": "optional"
        }
      ],
      "description": "\nApplies 'Sum' to each segment of input tensor. Segments ids can appear in\narbitrary order (unlike in SortedSegmentSum).\n\nSEGMENT_IDS is a vector that maps each of the first dimension slices of the\nDATA to a particular group (segment). Values belonging to the same segment are\naggregated together.\n\nIf `num_segments` argument is passed it would be used as a first dimension for\nthe output. Otherwise, it'd be dynamically calculated from as the max value of\nSEGMENT_IDS plus one. Other output dimensions are inherited from the input\ntensor.\n\nSummation is done element-wise across slices of the input tensor and doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Integer vector with the same length as the first dimension of DATA that maps each slice of DATA to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of equal to the number of segments.",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CreateTreeCursor",
    "schema": {
      "attributes": [
        {
          "description": "A list of strings each one representing a field of the dataset.",
          "name": "fields",
          "option": "optional"
        }
      ],
      "description": "\nCreates a cursor to iterate through a list of tensors, where some of those\ntensors contain the lengths in a nested schema. The schema is determined by\nthe `fields` arguments.\n\nFor example, to represent the following schema:\n\n  Struct(\n      a=Int(),\n      b=List(List(Int)),\n      c=List(\n          Struct(\n             c1=String,\n             c2=List(Int),\n          ),\n      ),\n  )\n\nthe field list will be:\n  [\n      \"a\",\n      \"b:lengths\",\n      \"b:values:lengths\",\n      \"b:values:values\",\n      \"c:lengths\",\n      \"c:c1\",\n      \"c:c2:lengths\",\n      \"c:c2:values\",\n  ]\n\nAnd for the following instance of the struct:\n\n  Struct(\n      a=3,\n      b=[[4, 5], [6, 7, 8], [], [9]],\n      c=[\n          Struct(c1='alex', c2=[10, 11]),\n          Struct(c1='bob', c2=[12]),\n      ],\n  )\n\nThe values of the fields will be:\n  {\n      \"a\": [3],\n      \"b:lengths\": [4],\n      \"b:values:lengths\": [2, 3, 0, 1],\n      \"b:values:values\": [4, 5, 6, 7, 8, 9],\n      \"c:lengths\": [2],\n      \"c:c1\": [\"alex\", \"bob\"],\n      \"c:c2:lengths\": [2, 1],\n      \"c:c2:values\", [10, 11, 12],\n  }\n\nIn general, every field name in the format \"{prefix}:lengths\" defines a domain\n\"{prefix}\", and every subsequent field in the format \"{prefix}:{field}\" will\nbe in that domain, and the length of the domain is provided for each entry of\nthe parent domain. In the example, \"b:lengths\" defines a domain of length 4, so\nevery field under domain \"b\" will have 4 entries.\nThe \"lengths\" field for a given domain must appear before any reference to\nthat domain.\n\nReturns a pointer to an instance of the Cursor, which keeps the current offset\non each of the domains defined by `fields`. Cursor also ensures thread-safety\nsuch that ReadNextBatch and ResetCursor can be used safely in parallel.\n\nA cursor does not contain data per se, so calls to ReadNextBatch actually need\nto pass a list of blobs containing the data to read for each one of the fields.\n",
      "outputs": [
        {
          "description": "A blob pointing to an instance of a new TreeCursor.",
          "name": "cursor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "UnsortedSegmentWeightedSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "HasElements",
    "schema": {
      "description": "\nThe *HasElements* op accepts a single or multiple input tensors, and produces a single boolean output $has\\_elements$. The output is *True* if and only if any of the input tensor has size > 0. Note, this op is the opposite of the *IsEmpty* op.\n\nGithub Links:\n\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/utility_ops.cc\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/utility_ops.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"HasElements\",\n    [\"tensor\"],\n    [\"has_elements\"],\n)\n\n// Use a not-empty tensor\nworkspace.FeedBlob(\"tensor\", np.random.randn(2, 2).astype(np.float32))\nprint(\"tensor:\\n\", workspace.FetchBlob(\"tensor\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"has_elements: \", workspace.FetchBlob(\"has_elements\"),\"\\n\")\n\n// Use an empty tensor\nworkspace.FeedBlob(\"tensor\", np.empty(0))\nprint(\"tensor:\\n\", workspace.FetchBlob(\"tensor\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"has_elements: \", workspace.FetchBlob(\"has_elements\"))\n\n```\n\n**Result**\n\n```\n\ntensor:\n [[ 0.6116506  -0.54433197]\n [ 0.19406661 -0.7338629 ]]\nhas_elements:  True\n\ntensor:\n []\nhas_elements:  False\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "Input data tensor to check for elements.",
          "name": "tensor"
        },
        {
          "description": "List of input data tensors to check for elements.",
          "name": "X1, X2, ..."
        }
      ],
      "outputs": [
        {
          "description": "Output scalar boolean tensor. True if input has size > 0.",
          "name": "has_elements"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8ConvTranspose",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "\nThe transposed convolution consumes an input vector, the filter blob, and\nthe bias blob, and computes the output. Note that other parameters, such as\nthe stride and kernel size, or the pads' sizes in each direction are not\nnecessary for input because they are provided by the\nConvTransposeUnpoolOpBase operator. Various dimension checks are done\nimplicitly, and the sizes are specified in the Input docs for this operator.\nAs is expected, the filter is deconvolved with a subset of the\nimage and the bias is added; this is done throughout the image data and the\noutput is computed. As a side note on the implementation layout:\nconv_transpose_op_impl.h is the templated implementation of the\nconv_transpose_op.h file, which is why they are separate files.\n  ",
      "inputs": [
        {
          "description": "Input data blob from previous layer; has size (N x H x W x C), where N is the batch size, C is the number of channels, and H and W are the height and width. Note that NHWC is supported now",
          "name": "X"
        },
        {
          "description": "The filter blob that will be used in the transposed convolution; has size (M x kH x kW x C), where C is the number of channels, and kH and kW are the height and width of the kernel.",
          "name": "filter"
        },
        {
          "description": "The 1D bias blob that is added through the convolution;has size (C). Optional, if not passed, will treat it as all 0.",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob that contains the result of the transposed convolution. The output dimensions are functions of the kernel size, stride size, and pad lengths.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LastNWindowCollector",
    "schema": {
      "attributes": [
        {
          "description": "The number of random samples to append for each positive samples",
          "name": "num_to_collect",
          "option": "optional"
        }
      ],
      "description": "\nCollect the last N rows from input data. The purpose is to keep track of data\naccross batches, so for example suppose the LastNWindowCollector is called\nsuccessively with the following input data\n\n  [1, 2, 3, 4]\n  [5, 6, 7]\n  [8, 9, 10, 11]\n\nAnd the number of items is set to 6, then the output after the 3rd call\nwill contain the following elements:\n\n  [6, 7, 8, 9, 10, 11]\n\nNo guarantee is made on the ordering of elements in input. So a valid value for\noutput could have been\n\n  [11, 10, 9, 8, 7, 6]\n\nAlso, this method works for any order tensor, treating the first dimension as\ninput rows and keeping the last N rows seen as input. So for instance:\n\n  [[1, 2], [2, 3], [3, 4], [4, 5]]\n  [[5, 6], [6, 7], [7, 8]]\n  [[8, 9], [9, 10], [10, 11], [11, 12]]\n\nA possible output would be\n\n  [[6, 7], [7, 8], [8, 9], [9, 10], [10, 11], [11, 12]]\n\nThis is not thread safe unless a mutex is given.\n",
      "inputs": [
        {
          "description": "The buffer for last-N record. Should be initialized to empty tensor",
          "name": "last-N buffer"
        },
        {
          "description": "The cursor pointing to the next position that should be replaced. Should be initialized to 0.",
          "name": "next cursor"
        },
        {
          "description": "tensor to collect from",
          "name": "DATA"
        },
        {
          "description": "(optional) mutex to use to make this thread-safe",
          "name": "MUTEX"
        },
        {
          "description": "",
          "name": "NUM_VISITED"
        }
      ],
      "outputs": [
        {
          "description": "Data stored in sessions",
          "name": "last-N buffer"
        },
        {
          "description": "Updated input cursor",
          "name": "next cursor"
        },
        {
          "description": "number of records seen so far",
          "name": "NUM_VISITED"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Bucketize",
    "schema": {
      "attributes": [
        {
          "description": "bucketization boundaries",
          "name": "boundaries",
          "option": "optional"
        }
      ],
      "description": "\nThis operator works as bucketize in tensorflow and digitize\nin numpy. It bucketizes the input 'X' based on argument 'boundaries'.\nFor each value x in input 'data', the operator returns index i given\nboundaries[i-1] < x <= boundaries[i].\nIf values in 'data' are beyond the bounds of boundaries, 0 or\nlen(boundaries) is returned as appropriate.\nThe boundaries need to be monotonically increasing.\nFor example\n\nIf data = [2, 4, 1] and boundaries = [0.1, 2.5], then\n\noutput = [1, 2, 1]\n\nIf data = [[2, 3], [4, 1], [2, 5]] and boundaries = [0.1, 2.5], then\n\noutput = [[1, 2], [2, 1], [1, 2]]\n\n",
      "inputs": [
        {
          "description": "input tensor",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "indices of bins given by boundaries to which each valuein data belongs",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HSoftmax",
    "schema": {
      "attributes": [
        {
          "description": "Serialized HierarchyProto string containing list of vocabulary words and their paths from root of hierarchy to the leaf",
          "name": "hierarchy",
          "option": "optional"
        }
      ],
      "description": "\nHierarchical softmax is an operator which approximates the softmax operator\nwhile giving significant training speed gains and reasonably comparable\nperformance. In this operator, instead of calculating the probabilities of all\nthe classes, we calculate the probability of each step in the path from root to\nthe target word in the hierarchy.\n\nThe operator takes a 2-D tensor (Tensor) containing a batch of layers, a\nset of parameters represented by the weight matrix and bias terms, and a 1-D\ntensor (Tensor) holding labels, or the indices of the target class. The\nhierarchy has to be specified as an argument to the operator.\n\nThe operator returns a 1-D tensor holding the computed log probability of the\ntarget class and a 2-D tensor of intermediate outputs (from the weight matrix\nand softmax from each step in the path from root to target class) which will be\nused by the gradient operator to compute gradients for all samples in the batch.\n",
      "inputs": [
        {
          "description": "Input data from previous layer",
          "name": "X"
        },
        {
          "description": "2D blob containing 'stacked' fully connected weight matrices. Each node in the hierarchy contributes one FC weight matrix if it has children nodes. Dimension is N*D, D is input dimension of data (X), N is sum of all output dimensions, or total number of nodes (excl root)",
          "name": "W"
        },
        {
          "description": "1D blob with N parameters",
          "name": "b"
        },
        {
          "description": "int word_id of the target word",
          "name": "labels"
        }
      ],
      "outputs": [
        {
          "description": "1-D of log probability outputs, one per sample",
          "name": "Y"
        },
        {
          "description": "Extra blob to store the intermediate FC and softmax outputs for each node in the hierarchical path of a word. The outputs from samples are stored in consecutive blocks in the forward pass and are used in reverse order in the backward gradientOp pass",
          "name": "intermediate_output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceFrontWeightedSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SpatialSoftmaxWithLoss",
    "schema": {
      "description": "\nCombined Spatial Softmax and Cross-Entropy loss operator.\nSimilar to SoftmaxWithLoss, this operator computes the spatial softmax\nnormalized values for each layer in the batch of the given input, after which\ncross-entropy loss is computed. This operator is numerically more stable than\nseparate Softmax and CrossEntropy ops. The inputs are a 2-D tensor\n(Tensor) of size (batch_size x input_feature_dimensions) and tensor of\nlabels (ground truth).\nOutput is tensor with the probability for each label in a pixel for each example\n(N x D x W x H) and averaged loss (scalar).\nFor spatial softmax, weighting is by x,y position of the input.\n",
      "inputs": [
        {
          "description": "Unscaled log probabilities",
          "name": "logits"
        },
        {
          "description": "Ground truth",
          "name": "labels"
        },
        {
          "description": "Optional blob to be used to weight the samples for the loss. With        spatial set, weighting is by x,y of the input",
          "name": "weight_tensor"
        }
      ],
      "outputs": [
        {
          "description": "Tensor with softmax cross entropy loss",
          "name": "softmax"
        },
        {
          "description": "Average loss",
          "name": "loss"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SafeDequeueBlobs",
    "schema": {
      "attributes": [
        {
          "description": "(default 1) If > 1, multiple records will be dequeued and tensors for each column will be concatenated. This requires all tensors in the records to be at least 1D, and to have the same inner dimensions.",
          "name": "num_records",
          "option": "optional"
        }
      ],
      "description": "\nDequeue the blobs from queue. When the queue is closed and empty, the output\nstatus will be set to true which can be used as exit criteria for execution\nstep.\nThe 1st input is the queue and the last output is the status. The rest are\ndata blobs.\n",
      "inputs": [
        {
          "description": "The shared pointer for the BlobsQueue",
          "name": "queue"
        }
      ],
      "outputs": [
        {
          "description": "The blob to store the dequeued data",
          "name": "blob"
        },
        {
          "description": "Is set to 0/1 depending on the success of dequeue",
          "name": "status"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Copy",
    "schema": {
      "description": "\nCopy input tensor into output, potentially across devices.\n\nGithub Links:\n\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/copy_op.cc\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/copy_op.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Copy\",\n    [\"input\"],\n    [\"output\"]\n)\n\nworkspace.FeedBlob(\"input\", np.random.rand(3,3))\nprint(\"input:\", workspace.FetchBlob(\"input\"))\nworkspace.RunOperatorOnce(op)\nprint(\"output:\", workspace.FetchBlob(\"output\"))\n\n```\n\n**Result**\n\n```\n\ninput:\n[[0.16826761 0.68168217 0.55196001]\n [0.19735483 0.34837823 0.69015595]\n [0.09448514 0.57390828 0.37097193]]\noutput:\n[[0.16826761 0.68168217 0.55196001]\n [0.19735483 0.34837823 0.69015595]\n [0.09448514 0.57390828 0.37097193]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor*): input tensor to copy",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor*): copy of input tensor",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AveragePool2DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentRangeLogMeanExp",
    "schema": {
      "description": "\nApplies 'LogMeanExp' to each segment of input tensor. In order to allow for more\nefficient implementation of 'LogMeanExp', the input segments have to be contiguous\nand non-empty.\n\nSEGMENT_IDS is a vector that maps each of the first dimension slices of the\nDATA to a particular group (segment). Values belonging to the same segment are\naggregated together.\n\nThe first dimension of the output is equal to the number of input segments,\ni.e. `SEGMENT_IDS[-1]+1`. Other dimensions are inherited from the input tensor.\n\nLogMeanExp computes the element-wise log of the mean of exponentials of input slices. Operation doesn't change the shape of individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor to be aggregated",
          "name": "DATA"
        },
        {
          "description": "Vector with the same length as the first dimension of DATA and values in the range 0..K-1 and in increasing order that maps each slice of DATA to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated tensor with the first dimension of K and the other dimentsions inherited from DATA",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MergeMultiListFeatureTensors",
    "schema": {
      "description": "Merge given multi-feature tensors with list features into one.\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": ".lengths",
          "name": "in1_lengths"
        },
        {
          "description": ".keys",
          "name": "in1_keys"
        },
        {
          "description": ".values.lengths",
          "name": "in1_values_lengths"
        },
        {
          "description": ".values.values",
          "name": "in1_values_values"
        }
      ],
      "outputs": [
        {
          "description": ".lengths",
          "name": "out_lengths"
        },
        {
          "description": ".keys",
          "name": "out_keys"
        },
        {
          "description": ".values.lengths",
          "name": "out_values_lengths"
        },
        {
          "description": ".values.values",
          "name": "out_values_values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CountUp",
    "schema": {
      "description": "\nIncreases count value by 1 and outputs the previous value atomically.\n\n  Github Links:\n  - https://github.com/pytorch/pytorch/blob/master/caffe2/operators/counter_ops.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\ncreatecounter_op = core.CreateOperator(\n    \"CreateCounter\",\n    [],\n    [\"counter\"],\n    init_count=5\n)\n\nretrievecount_op = core.CreateOperator(\n    \"RetrieveCount\",\n    [\"counter\"],\n    [\"count\"]\n)\n\ncheckcounterdone_op = core.CreateOperator(\n    \"CheckCounterDone\",\n    [\"counter\"],\n    [\"done\"]\n)\n\ncountup_op = core.CreateOperator(\n    \"CountUp\",\n    [\"counter\"],\n    [\"previous_count\"],\n)\n\ncountdown_op = core.CreateOperator(\n    \"CountDown\",\n    [\"counter\"],\n    [\"done\"],\n)\n\nresetcounter_op = core.CreateOperator(\n    \"ResetCounter\",\n    [\"counter\"],\n    [\"previous_count\"],\n    init_count=3\n)\n\n\n// Create counter\nworkspace.RunOperatorOnce(createcounter_op)\nprint(\"'counter' pointer:\", workspace.FetchBlob(\"counter\"))\n\n\n// Retrieve initial counter value\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"Initial 'count':\", workspace.FetchBlob(\"count\"))\n\n\n// Check if counter is done\nworkspace.RunOperatorOnce(checkcounterdone_op)\nprint(\"Initial 'done' value:\", workspace.FetchBlob(\"done\"))\n\n\n// Test CountUp operator\nprint(\"\\nTesting CountUp operator...\")\nfor i in range(5):\n    workspace.RunOperatorOnce(countup_op)\n    print(\"'previous_count' after CountUp:\", workspace.FetchBlob(\"previous_count\"))\n\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"'count' value after CountUp test:\", workspace.FetchBlob(\"count\"))\n\n\n// Test CountDown operator\nprint(\"\\nTesting CountDown operator...\")\nfor i in range(11):\n    workspace.RunOperatorOnce(countdown_op)\n    workspace.RunOperatorOnce(retrievecount_op)\n    print(\"'count' value after CountDown: {}\\t'done' value: {}\".format(workspace.FetchBlob(\"count\"), workspace.FetchBlob(\"done\")))\n```\n\n**Result**\n\n```\n'counter' pointer: counter, a C++ native class of type std::__1::unique_ptr<caffe2::Counter<long long>, std::__1::default_delete<caffe2::Counter<long long> > >.\nInitial 'count': 5\nInitial 'done' value: False\n\nTesting CountUp operator...\n'previous_count' after CountUp: 5\n'previous_count' after CountUp: 6\n'previous_count' after CountUp: 7\n'previous_count' after CountUp: 8\n'previous_count' after CountUp: 9\n'count' value after CountUp test: 10\n\nTesting CountDown operator...\n'count' value after CountDown: 9        'done' value: False\n'count' value after CountDown: 8        'done' value: False\n'count' value after CountDown: 7        'done' value: False\n'count' value after CountDown: 6        'done' value: False\n'count' value after CountDown: 5        'done' value: False\n'count' value after CountDown: 4        'done' value: False\n'count' value after CountDown: 3        'done' value: False\n'count' value after CountDown: 2        'done' value: False\n'count' value after CountDown: 1        'done' value: False\n'count' value after CountDown: 0        'done' value: False\n'count' value after CountDown: -1        'done' value: True\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<ptr>`)* A blob pointing to an instance of a counter.",
          "name": "counter"
        }
      ],
      "outputs": [
        {
          "description": "*(type: int)* Count value BEFORE this operation.",
          "name": "previous_count"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MergeSingleScalarFeatureTensorsGradient",
    "schema": {
      "description": "Explode multi-feature tensor of scalar features into one or moresingle-feature tensors\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": ".presence",
          "name": "in1_presence"
        },
        {
          "description": ".values_grad",
          "name": ".values_grad"
        }
      ],
      "outputs": [
        {
          "description": "_grad of inputs",
          "name": "in1_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GFtrl",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Acos",
    "schema": {
      "description": "\nCalculates the arccosine of the given input tensor, element-wise.\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The arccosine of the input tensor computed element-wise",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SoftsignGradient",
    "schema": {
      "description": "\nCalculates the softsign gradient (sgn(x)/(1+|x|)^2) of the given input tensor\nelement-wise.\n",
      "inputs": [
        {
          "description": "1-D input tensor",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The softsign gradient (sgn(x)/(1+|x|)^2) values of the input tensor computed element-wise",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MergeDim",
    "schema": {
      "description": "\nMerge first two dimensions in a single dimension with size dim(0) * dim(1).\n",
      "inputs": [
        {
          "description": "An input tensor.",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "Reshaped tensor.",
          "name": "reshaped"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8ResizeNearest",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        },
        {
          "description": "Scale along width dimension",
          "name": "width_scale",
          "option": "optional"
        },
        {
          "description": "Scale along height dimension",
          "name": "height_scale",
          "option": "optional"
        },
        {
          "description": "Output dimensions (HxW). If specified this takes precedence over scale values.",
          "name": "output_size",
          "option": "optional"
        }
      ],
      "description": "\nResizes the spatial dimensions of the input using nearest neighbor\ninterpolation. The `width_scale` and `height_scale` arguments\ncontrol the size of the output, which is given by:\noutput_width = floor(input_width * width_scale)\noutput_height = floor(output_height * height_scale)\n",
      "inputs": [
        {
          "description": "Input Int8 tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output Int8 tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LC",
    "schema": {
      "description": "\nThe locally connected operator consumes an input vector, a filter blob\nand a bias blob and computes the output. \nNote that other parameters, such as the stride and\nkernel size, or the pads' sizes in each direction are not necessary for input\nbecause they are provided by the ConvPoolOpBase operator. Various dimension\nchecks are done implicitly, and the sizes are specified in the Input docs for\nthis operator. As is expected, the filter is locally connected with a subset of\nthe image and the bias is added; this is done throughout the image data and the\noutput is computed. As a side note on the implementation layout:\nlocally_connected_op_impl.h is the templated implementation of the\nlocally_connected_op.h file, which is why they are separate files.\n",
      "inputs": [
        {
          "description": null,
          "name": null
        },
        {
          "description": "The filter blob that will be used in the locally connected op; has size (YH * YW * M x C x kH x kW) if order == NCHW else (YH * YW * M  * KH * KW * C), where YH and YW are the height and width of the output image, C is the number of channels, and kH and kW are the height and width of the kernel.",
          "name": "filter"
        },
        {
          "description": "The 1D bias blob that is added through the locally connected op; has size (YH * YW * M).",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob that contains the result of the locally connected op.The output dimensions are functions of the kernel size, stride size, and pad lengths.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AcosGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "IndexHash",
    "schema": {
      "attributes": [
        {
          "description": "seed for the hash function",
          "name": "seed",
          "option": "optional"
        },
        {
          "description": "must be > 0, hashed ids will be modulo this number",
          "name": "modulo",
          "option": "optional"
        }
      ],
      "description": "\nThis operator translates a list of indices into a list of hashed indices.\nA seed can be fed as an argument to change the behavior of the hash function.\nIf a modulo is specified, all the hashed indices will be modulo the\nspecified number. All input and output indices are enforced to be positive.\n",
      "inputs": [
        {
          "description": "Input feature indices.",
          "name": "Indices"
        }
      ],
      "outputs": [
        {
          "description": "Hashed feature indices.",
          "name": "HashedIndices"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GroupNorm",
    "schema": {
      "attributes": [
        {
          "description": "(int) default 32; number of groups used by GN.",
          "name": "num_groups",
          "option": "optional"
        },
        {
          "description": "(float) default 1e-5; small constant added to var.",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\nGroup Normalization (GN) operation: https://arxiv.org/abs/1803.08494\n",
      "inputs": [
        {
          "description": ">=4D feature map input of shape (N, C, H, W) or (N, C, T, H, W)",
          "name": "X"
        },
        {
          "description": "The scale as a 1-dimensional tensor of size C to be applied to the output.",
          "name": "gamma"
        },
        {
          "description": "The bias as a 1-dimensional tensor of size C to be applied to the output.",
          "name": "beta"
        }
      ],
      "outputs": [
        {
          "description": "The output >=4-dimensional tensor of the same shape as X.",
          "name": "Y"
        },
        {
          "description": "The mean of shape (N, G). For backward usage or reference. Cannot be used as activations.",
          "name": "mean"
        },
        {
          "description": "The std of shape (N, G). For backward usage or reference. Cannot be used as activations.",
          "name": "std"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ChannelShuffleGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "DestroyCommonWorld",
    "schema": {
      "description": "Closes all connections managed by a common world.",
      "inputs": [
        {
          "description": "The common world to be destroyed.",
          "name": "common_world"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Floor",
    "schema": {
      "description": "\nElement-wise application of the floor function ($y=floor(x)$) to the input\ntensor `X`. Output tensor shape is the same as the input tensor. This\noperator can be used in an in-place fashion by using the same input blob as the\noutput blob.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/floor_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Floor\",\n    [\"X\"],\n    [\"X\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.uniform(-10, 10, (5,5))).astype(np.float32))\nprint(\"X before running op:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"X after running op:\", workspace.FetchBlob(\"X\"))\n\n```\n\n**Result**\n\n```\n\nX before running op:\n[[ 3.813361   -1.319647    5.2089314  -4.931328    0.6218652 ]\n [ 7.2757645   5.5552588   5.785643   -2.4790506  -0.41400087]\n [ 1.1541046  -6.933266    3.3754056   1.6569928  -1.7670316 ]\n [-3.4932013   4.891472    1.5530115  -3.2443287  -4.605099  ]\n [-4.574543   -7.360948    5.91305    -8.196495   -5.357458  ]]\nX after running op:\n[[ 3. -2.  5. -5.  0.]\n [ 7.  5.  5. -3. -1.]\n [ 1. -7.  3.  1. -2.]\n [-4.  4.  1. -4. -5.]\n [-5. -8.  5. -9. -6.]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsPartition",
    "schema": {
      "attributes": [
        {
          "description": "(int, default 0) If set, the operator transforms the first tensor values as floor(X_ij / num_partitions)",
          "name": "pack_first_input",
          "option": "optional"
        }
      ],
      "description": "\nLengthsPartition splits the input int tensor into multiple ones according to the\nsecond tensor. The first dimension is expected to be the tensor that describes\nlengths of the elements.\n\nTakes the second input and partitions it to shards according to the remainder of\nvalues modulo the number of partitions. It requires the second tensor to be\na 1D-tensor of the integral type. The first tensor should be 1D-tensor of int32\nthat would represent the lengths of the elements in the input. The number of\npartitions is derived as (num_output / num_input).\n\nIf additional inputs are present they must have the same shape as the first\ninput, optionally with extra trailing dimensions. They will be partitioned\naccordingly to the first input.\n\nOptional arg 'pack_first_input' transforms the first tensor values as\nX_ij / num_partitions.\n\nOutputs are ordered as\nX_0_part_0, X_1_part_0, ..., X_N-1_part_0, X_0_part_1, ..., X_N-1_part_K-1\n",
      "inputs": [
        {
          "description": "Input tensor containing data to be partitioned. The number of input tensors might be greater than 1 but must have the same shape as the previous tensors.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Output Partitions. The number of output tensors has to be a multiple of the number of input tensors.",
          "name": "partitions"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CreateCounter",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Initial count for the counter, must be >= 0.",
          "name": "init_count",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nCreates a count-down counter with initial value specified by the `init_count`\nargument.\n\n\n  Github Links:\n  - https://github.com/pytorch/pytorch/blob/master/caffe2/operators/counter_ops.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\ncreatecounter_op = core.CreateOperator(\n    \"CreateCounter\",\n    [],\n    [\"counter\"],\n    init_count=5\n)\n\nretrievecount_op = core.CreateOperator(\n    \"RetrieveCount\",\n    [\"counter\"],\n    [\"count\"]\n)\n\ncheckcounterdone_op = core.CreateOperator(\n    \"CheckCounterDone\",\n    [\"counter\"],\n    [\"done\"]\n)\n\ncountup_op = core.CreateOperator(\n    \"CountUp\",\n    [\"counter\"],\n    [\"previous_count\"],\n)\n\ncountdown_op = core.CreateOperator(\n    \"CountDown\",\n    [\"counter\"],\n    [\"done\"],\n)\n\nresetcounter_op = core.CreateOperator(\n    \"ResetCounter\",\n    [\"counter\"],\n    [\"previous_count\"],\n    init_count=3\n)\n\n\n// Create counter\nworkspace.RunOperatorOnce(createcounter_op)\nprint(\"'counter' pointer:\", workspace.FetchBlob(\"counter\"))\n\n\n// Retrieve initial counter value\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"Initial 'count':\", workspace.FetchBlob(\"count\"))\n\n\n// Check if counter is done\nworkspace.RunOperatorOnce(checkcounterdone_op)\nprint(\"Initial 'done' value:\", workspace.FetchBlob(\"done\"))\n\n\n// Test CountUp operator\nprint(\"\\nTesting CountUp operator...\")\nfor i in range(5):\n    workspace.RunOperatorOnce(countup_op)\n    print(\"'previous_count' after CountUp:\", workspace.FetchBlob(\"previous_count\"))\n\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"'count' value after CountUp test:\", workspace.FetchBlob(\"count\"))\n\n\n// Test CountDown operator\nprint(\"\\nTesting CountDown operator...\")\nfor i in range(11):\n    workspace.RunOperatorOnce(countdown_op)\n    workspace.RunOperatorOnce(retrievecount_op)\n    print(\"'count' value after CountDown: {}\\t'done' value: {}\".format(workspace.FetchBlob(\"count\"), workspace.FetchBlob(\"done\")))\n```\n\n**Result**\n\n```\n'counter' pointer: counter, a C++ native class of type std::__1::unique_ptr<caffe2::Counter<long long>, std::__1::default_delete<caffe2::Counter<long long> > >.\nInitial 'count': 5\nInitial 'done' value: False\n\nTesting CountUp operator...\n'previous_count' after CountUp: 5\n'previous_count' after CountUp: 6\n'previous_count' after CountUp: 7\n'previous_count' after CountUp: 8\n'previous_count' after CountUp: 9\n'count' value after CountUp test: 10\n\nTesting CountDown operator...\n'count' value after CountDown: 9        'done' value: False\n'count' value after CountDown: 8        'done' value: False\n'count' value after CountDown: 7        'done' value: False\n'count' value after CountDown: 6        'done' value: False\n'count' value after CountDown: 5        'done' value: False\n'count' value after CountDown: 4        'done' value: False\n'count' value after CountDown: 3        'done' value: False\n'count' value after CountDown: 2        'done' value: False\n'count' value after CountDown: 1        'done' value: False\n'count' value after CountDown: 0        'done' value: False\n'count' value after CountDown: -1        'done' value: True\n```\n\n</details>\n\n",
      "outputs": [
        {
          "description": "*(type: Tensor`<ptr>`)* A blob pointing to an instance of a new counter.",
          "name": "counter"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MarginRankingCriterion",
    "schema": {
      "attributes": [
        {
          "description": "The margin value as a float. Default is 1.0.",
          "name": "margin",
          "option": "optional"
        }
      ],
      "description": "\nMarginRankingCriterion takes two input data X1 (Tensor),\nX2 (Tensor), and label Y (Tensor) to produce the\nloss (Tensor) where the loss function,\nloss(X1, X2, Y) = max(0, -Y * (X1 - X2) + margin), is applied to\nthe tensor elementwise.\n\nIf y == 1 then it assumed the first input should be ranked higher\n(have a larger value) than the second input, and vice-versa for\ny == -1.\n",
      "inputs": [
        {
          "description": "The left input vector as a 1-dim TensorCPU.",
          "name": "X1"
        },
        {
          "description": "The right input vector as a 1-dim TensorCPU.",
          "name": "X2"
        },
        {
          "description": "The label as a 1-dim TensorCPU with int value of 1 or -1.",
          "name": "Y"
        }
      ],
      "outputs": [
        {
          "description": "The output loss with the same dimensionality as X1.",
          "name": "loss"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MergeIdLists",
    "schema": {
      "description": "\nMergeIdLists: Merge multiple ID_LISTs into a single ID_LIST.\n\nAn ID_LIST is a list of IDs (may be ints, often longs) that represents a single\nfeature. As described in https://caffe2.ai/docs/sparse-operations.html, a batch\nof ID_LIST examples is represented as a pair of lengths and values where the\n`lengths` (int32) segment the `values` or ids (int32/int64) into examples.\n\nGiven multiple inputs of the form lengths_0, values_0, lengths_1, values_1, ...\nwhich correspond to lengths and values of ID_LISTs of different features, this\noperator produces a merged ID_LIST that combines the ID_LIST features. The\nfinal merged output is described by a lengths and values vector.\n\nWARNING: The merge makes no guarantee about the relative order of ID_LISTs\nwithin a batch. This can be an issue if ID_LIST are order sensitive.\n",
      "inputs": [
        {
          "description": "Lengths of the ID_LISTs batch for first feature",
          "name": "lengths_0"
        },
        {
          "description": "Values of the ID_LISTs batch for first feature",
          "name": "values_0"
        }
      ],
      "outputs": [
        {
          "description": "Lengths of the merged ID_LISTs batch",
          "name": "merged_lengths"
        },
        {
          "description": "Values of the merged ID_LISTs batch",
          "name": "merged_values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SumElements",
    "schema": {
      "attributes": [
        {
          "description": "(*bool*): set to True to compute the average of the elements rather than the sum",
          "name": "average",
          "option": "optional"
        }
      ],
      "description": "\nSums the elements of the input tensor. Tensor type must be float32.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduction_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nsum_op = core.CreateOperator(\n    \"SumElements\",\n    [\"X\"],\n    [\"Y\"]\n)\n\navg_op = core.CreateOperator(\n    \"SumElements\",\n    [\"X\"],\n    [\"Y\"],\n    average=True\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(3,3)).astype(np.float32))\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(sum_op)\nprint(\"Y (sum_op):\", workspace.FetchBlob(\"Y\"))\nworkspace.RunOperatorOnce(avg_op)\nprint(\"Y (avg_op):\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[7. 2. 5.]\n [9. 4. 2.]\n [1. 2. 5.]]\nY (sum_op): 37.0\nY (avg_op): 4.111111\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): blob pointing to an instance of a counter",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): Scalar tensor containing the sum (or average)",
          "name": "sum"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ThresholdedReluGradient",
    "schema": {
      "description": "\nThresholdedReluGradient takes both Y and dY and uses this to update dX\naccording to the chain rule and derivatives of the rectified linear function.\n",
      "support_level": "default"
    }
  },
  {
    "name": "GivenTensorInt64Fill",
    "schema": {
      "attributes": [
        {
          "description": "The value for the elements of the output tensor.",
          "name": "values"
        },
        {
          "description": "The shape of the output tensor.Cannot set the shape argument and pass in an input at the same time.",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "The additional dimensions appended at the end of the shape indicatedby the input blob.Cannot set the extra_shape argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional"
        },
        {
          "description": "1D tensor containing the desired output shape. First input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsSum",
    "schema": {
      "description": "\nPulls in slices of the input tensor, groups them into segments and applies\n'Sum' to each segment. Segments are defined by their LENGTHS.\n\nThis op is basically Gather and LengthsSum fused together.\n\nINDICES should contain integers in range 0..N-1 where N is the first dimension\nof DATA. INDICES represent which slices of DATA need to be pulled in.\n\nLENGTHS is a vector that defines slice sizes by first dimension of DATA. Values\nbelonging to the same segment are aggregated together. sum(LENGTHS) has\nto match INDICES size.\n\nThe first dimension of the output is equal to the number of input segment,\ni.e. `len(LENGTHS)`. Other dimensions are inherited from the input tensor.\n\nSummation is done element-wise across slices of the input tensor and doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Non negative vector with sum of elements equal to INDICES length",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of K (the number of segments).",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CountDown",
    "schema": {
      "description": "\nIf the internal count value > 0, decreases count value by 1 and outputs False,\notherwise outputs True.\n\n  Github Links:\n  - https://github.com/pytorch/pytorch/blob/master/caffe2/operators/counter_ops.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\ncreatecounter_op = core.CreateOperator(\n    \"CreateCounter\",\n    [],\n    [\"counter\"],\n    init_count=5\n)\n\nretrievecount_op = core.CreateOperator(\n    \"RetrieveCount\",\n    [\"counter\"],\n    [\"count\"]\n)\n\ncheckcounterdone_op = core.CreateOperator(\n    \"CheckCounterDone\",\n    [\"counter\"],\n    [\"done\"]\n)\n\ncountup_op = core.CreateOperator(\n    \"CountUp\",\n    [\"counter\"],\n    [\"previous_count\"],\n)\n\ncountdown_op = core.CreateOperator(\n    \"CountDown\",\n    [\"counter\"],\n    [\"done\"],\n)\n\nresetcounter_op = core.CreateOperator(\n    \"ResetCounter\",\n    [\"counter\"],\n    [\"previous_count\"],\n    init_count=3\n)\n\n\n// Create counter\nworkspace.RunOperatorOnce(createcounter_op)\nprint(\"'counter' pointer:\", workspace.FetchBlob(\"counter\"))\n\n\n// Retrieve initial counter value\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"Initial 'count':\", workspace.FetchBlob(\"count\"))\n\n\n// Check if counter is done\nworkspace.RunOperatorOnce(checkcounterdone_op)\nprint(\"Initial 'done' value:\", workspace.FetchBlob(\"done\"))\n\n\n// Test CountUp operator\nprint(\"\\nTesting CountUp operator...\")\nfor i in range(5):\n    workspace.RunOperatorOnce(countup_op)\n    print(\"'previous_count' after CountUp:\", workspace.FetchBlob(\"previous_count\"))\n\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"'count' value after CountUp test:\", workspace.FetchBlob(\"count\"))\n\n\n// Test CountDown operator\nprint(\"\\nTesting CountDown operator...\")\nfor i in range(11):\n    workspace.RunOperatorOnce(countdown_op)\n    workspace.RunOperatorOnce(retrievecount_op)\n    print(\"'count' value after CountDown: {}\\t'done' value: {}\".format(workspace.FetchBlob(\"count\"), workspace.FetchBlob(\"done\")))\n```\n\n**Result**\n\n```\n'counter' pointer: counter, a C++ native class of type std::__1::unique_ptr<caffe2::Counter<long long>, std::__1::default_delete<caffe2::Counter<long long> > >.\nInitial 'count': 5\nInitial 'done' value: False\n\nTesting CountUp operator...\n'previous_count' after CountUp: 5\n'previous_count' after CountUp: 6\n'previous_count' after CountUp: 7\n'previous_count' after CountUp: 8\n'previous_count' after CountUp: 9\n'count' value after CountUp test: 10\n\nTesting CountDown operator...\n'count' value after CountDown: 9        'done' value: False\n'count' value after CountDown: 8        'done' value: False\n'count' value after CountDown: 7        'done' value: False\n'count' value after CountDown: 6        'done' value: False\n'count' value after CountDown: 5        'done' value: False\n'count' value after CountDown: 4        'done' value: False\n'count' value after CountDown: 3        'done' value: False\n'count' value after CountDown: 2        'done' value: False\n'count' value after CountDown: 1        'done' value: False\n'count' value after CountDown: 0        'done' value: False\n'count' value after CountDown: -1        'done' value: True\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<ptr>`)* A blob pointing to an instance of a counter.",
          "name": "counter"
        }
      ],
      "outputs": [
        {
          "description": "*(type: bool)* False unless the internal count is zero.",
          "name": "done"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReciprocalGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Sqr",
    "schema": {
      "description": "\nPerforms element-wise squaring ($x^2$) of input tensor.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/sqr_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Sqr\",\n    [\"X\"],\n    [\"Y\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.randint(10, size=(3,3))).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[4. 6. 2.]\n [0. 1. 6.]\n [9. 2. 7.]]\nY:\n[[16. 36.  4.]\n [ 0.  1. 36.]\n [81.  4. 49.]]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StoreWait",
    "schema": {
      "attributes": [
        {
          "description": "names of the blobs to wait for (optional)",
          "name": "blob_names",
          "option": "optional"
        }
      ],
      "description": "\nWait for the specified blob names to be set. The blob names can be passed\neither as an input blob with blob names or as an argument.\n",
      "inputs": [
        {
          "description": "unique_ptr<StoreHandler>",
          "name": "handler"
        },
        {
          "description": "names of the blobs to wait for (optional)",
          "name": "names"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ColwiseMax",
    "schema": {
      "description": "\nCompute column-wise max reduction of the input tensor. This op takes one input, $X$, of shape $BxMxN$, where $B$ is the batch size, $M$ is number of rows, and $N$ is number of columns. The output of this op, $Y$, is a matrix of shape $BxN$, with one row for each element of the batch, and the same number of columns as the input tensor.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduction_ops.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduction_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ColwiseMax\",\n    [\"X\"],\n    [\"Y\"]\n)\n\n// Create X, simulating a batch of 2, 4x4 matricies\nX = np.random.randint(0,high=20,size=(2,4,4))\nprint(\"X:\\n\",X)\n\n// Feed X into workspace\nworkspace.FeedBlob(\"X\", X.astype(np.float32))\n\n// Run op\nworkspace.RunOperatorOnce(op)\n\n// Collect Output\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[[17 15  2  6]\n  [ 8 12  6  0]\n  [ 6  9  7  3]\n  [ 4 13 16 13]]\n\n [[ 0  3  4 12]\n  [18  1 17 12]\n  [ 7 17 13 14]\n  [12 17  2  1]]]\nY:\n [[17. 15. 16. 13.]\n [18. 17. 17. 14.]]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "A tensor of dimensions $B x M x N$ to compute columnwise-max. Here, $B$ is batch size, and $M$ and $N$ are the number of rows and columns of each element of the batch, respectively.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "The output tensor of shape $B x N$, where each row represents the column-wise maximums for that element of the input batch.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LogFatal",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "StringIndexCreate",
    "schema": {
      "attributes": [
        {
          "description": "Max number of elements, including the zero entry.",
          "name": "max_elements",
          "option": "optional"
        }
      ],
      "description": "\nCreates a dictionary that maps string keys to consecutive integers\nfrom 1 to max_elements. Zero is reserved for unknown keys.\n",
      "outputs": [
        {
          "description": "Pointer to an Index instance.",
          "name": "handle"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CopyRowsToTensor",
    "schema": {
      "description": "\n      This operator takes in a 2d tensor, a list of indices, and a 1d tensor\n      with the same width of the 2d tensor. It will replace the rows in 2d\n      tensor specified in indices with the 2d tensor. The operator does an\n      in-place change to the input tensor.\n      Example:\n        INPUT_TENSOR = [[1, 2], [3, 4], [5, 6]]\n        INDICES = [1]\n        ROW = [9, 0]\n        OUTPUT_TENSOR = [[1, 2], [9, 0], [5, 6]]\n      ",
      "inputs": [
        {
          "description": "Input tensor needs to be modified.",
          "name": "input_tensor"
        },
        {
          "description": "Indices of rows need to be copied",
          "name": "indices"
        },
        {
          "description": "1-d tensor that is going to replace the rows",
          "name": "row"
        }
      ],
      "outputs": [
        {
          "description": "updated tensor",
          "name": "output_tensor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MakeTwoClass",
    "schema": {
      "description": "\nGiven a vector of probabilities, this operator transforms this into a 2-column\n matrix with complimentary probabilities for binary classification. In explicit\n terms, given the vector X, the output Y is vstack(1 - X, X).\n  ",
      "inputs": [
        {
          "description": "Input vector of probabilities",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "2-column matrix with complimentary probabilities of X for binary classification",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Snapshot",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "NegateGradient",
    "schema": {
      "description": "\nNegagteGradient operator in forward pass simply copies input to the\noutput, and in backward pass, flips the sign of the output gradient\n",
      "support_level": "default"
    }
  },
  {
    "name": "Not",
    "schema": {
      "description": "\nPerforms element-wise negation on input tensor `X`.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n\"Not\",\n[\"X\"],\n[\"Y\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.rand(3, 3) > 0.5))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[ True False False]\n[False False False]\n[ True  True  True]]\nY:\n[[False  True  True]\n[ True  True  True]\n[False False False]]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "*(Tensor`<bool>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(Tensor`<bool>`)* Negated output tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PrependDim",
    "schema": {
      "attributes": [
        {
          "description": "Size of the dimension to prepend.",
          "name": "dim_size",
          "option": "optional"
        }
      ],
      "description": "\nReshape the tensor by prepending a dimension of fixed size and dividing the\nsize of the next dimension by that amount.\n",
      "inputs": [
        {
          "description": "An input tensor.",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "Reshaped tensor.",
          "name": "reshaped"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SendTensor",
    "schema": {
      "attributes": [
        {
          "description": "The rank to send the tensor to.",
          "name": "dst",
          "option": "optional"
        },
        {
          "description": "(int) a tag to send the tensor with.",
          "name": "tag",
          "option": "optional"
        },
        {
          "description": "(bool) if set, only send the content and assume that the receiver has already known the tensor's shape and information.",
          "name": "raw_buffer",
          "option": "optional"
        }
      ],
      "description": "\nSends the tensor to another node.\n",
      "inputs": [
        {
          "description": "The common world.",
          "name": "comm_world"
        },
        {
          "description": "A tensor to be allgathered.",
          "name": "X"
        },
        {
          "description": "An int CPUtensor of size 1 specifying the rank. If given, this overrides the 'to' argument of the op.",
          "name": "dst"
        },
        {
          "description": "An int CPUtensor of size 1 specifying the tag to send the tensor with. This overrides the 'tag' argument of the op.",
          "name": "tag"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "InferenceLSTM",
    "schema": {
      "attributes": [
        {
          "description": "(*long*): number of layers in the lstm stack",
          "name": "num_layers",
          "option": "optional"
        },
        {
          "description": "(*bool*): whether the cells have biases or not",
          "name": "has_biases",
          "option": "optional"
        },
        {
          "description": "(*bool*): whether the batch is at dim 0",
          "name": "batch_first",
          "option": "optional"
        },
        {
          "description": "(*bool*): if bidirectional",
          "name": "bidirectional",
          "option": "optional"
        }
      ],
      "description": null,
      "outputs": [
        {
          "description": "the output of the last layer of lstm",
          "name": "output"
        },
        {
          "description": "hidden state at t = seq_len",
          "name": "hidden"
        },
        {
          "description": "cell state at t = seq_len",
          "name": "cell"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SumElementsInt",
    "schema": {
      "description": "Sums the integer elements of the input tensor.",
      "inputs": [
        {
          "description": "Tensor to sum up",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Scalar sum",
          "name": "sum"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseAdagrad",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\nGiven inputs (param, moment, indices, grad, lr), runs the dense AdaGrad\nupdate on (param, grad, moment[indices], lr), and returns (new_param,\nnew_moment) as in the dense case.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Moment history",
          "name": "moment"
        },
        {
          "description": "Sparse indices",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated moment",
          "name": "output_moment_1"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8AveragePoolRelu",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "AveragePool \nconsumes an input blob X and applies average pooling across the\nthe blob according to kernel sizes, stride sizes, and pad lengths defined by the\nConvPoolOpBase operator. Average pooling consisting of averaging all values of a\nsubset of the input tensor according to the kernel size and downsampling the\ndata into the output blob Y for further processing.\n",
      "inputs": [
        {
          "description": "Input data tensor from the previous operator; dimensions depend on whether the NCHW or NHWC operators are being used. For example, in the former, the input has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. The corresponding permutation of dimensions is used in the latter case.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output data tensor from average pooling across the input tensor. Dimensions will vary based on various kernel, stride, and pad sizes. Output will go through rectified linear function, where y = max(0, x).",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SegmentIdsToRanges",
    "schema": {
      "description": "\nTransfers a vector of segment ids to a vector of segment ranges. This operation\nsupports non-consecutive segment ids. Segments not appearing in the input vector\nwill have length 0. If the second input is provided, the number of segments =\nthe size of its first dimension. Otherwise, the number of segments = the last\nindex in the first input vector + 1.\n",
      "inputs": [
        {
          "description": "1-D int32_t or int64_t tensor of segment ids",
          "name": "segment_ids"
        },
        {
          "description": "if provided, number of segments = the size of its first dimension",
          "name": "data (optional)"
        }
      ],
      "outputs": [
        {
          "description": "1-D int64_t tensor of segment lengths",
          "name": "lengths"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CreateCommonWorld",
    "schema": {
      "attributes": [
        {
          "description": "(int) size of the common world.",
          "name": "size",
          "option": "optional"
        },
        {
          "description": "(int) rank of this node in the common world.",
          "name": "rank",
          "option": "optional"
        }
      ],
      "description": "\nCreates a common world for communication operators.\n",
      "inputs": [
        {
          "description": "Key/value handler for rendezvous (optional).",
          "name": "kv_handler"
        }
      ],
      "outputs": [
        {
          "description": "A common world for collective operations.",
          "name": "comm_world"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SliceGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseUnsortedSegmentWeightedSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "DBExists",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "If set to non-zero, save the db directly to the path specified by the `db` arg. If not set (default), prepend the path of the current root folder of the workspace to the path specified by the `db` arg.",
          "name": "absolute_path",
          "option": "optional",
          "type": "int64"
        },
        {
          "description": "Path to the db in question; see the `absolute_path` arg details for options regarding the current root folder of the workspace.",
          "name": "db_name",
          "option": "optional",
          "type": "string"
        },
        {
          "description": "Type of db to save (options: \"lmdb\", \"leveldb\", \"minidb\").",
          "name": "db_type",
          "option": "optional",
          "type": "string"
        }
      ],
      "description": "\nChecks if the db described by the arguments exists.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/load_save_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"DBExists\",\n    [],\n    [\"exists\"],\n    db_name=\"test_db\",\n    db_type=\"leveldb\",\n)\n\nworkspace.RunOperatorOnce(op)\nprint(\"exists:\", workspace.FetchBlob(\"exists\"))\n\n```\n\n</details>\n\n",
      "outputs": [
        {
          "description": "*(type: Tensor`<bool>`)* Scalar boolean output tensor. True if the db exists, else false.",
          "name": "exists"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReceiveTensor",
    "schema": {
      "attributes": [
        {
          "description": "(int) he rank to receive the tensor from.",
          "name": "src",
          "option": "optional"
        },
        {
          "description": "(int) a tag to receive the tensor with.",
          "name": "tag",
          "option": "optional"
        },
        {
          "description": "(bool) if set, only send the content and assume that the receiver has already known the tensor's shape and information.",
          "name": "raw_buffer",
          "option": "optional"
        }
      ],
      "description": "\nReceives the tensor from another node.\n",
      "inputs": [
        {
          "description": "The common world.",
          "name": "comm_world"
        },
        {
          "description": "In-place output. If raw_buffer is specified, Y should have pre-allocated data and type..",
          "name": "Y"
        },
        {
          "description": "An int CPUtensor of size 1 specifying the rank. If given, this overrides the 'from' argument of the op.",
          "name": "src"
        },
        {
          "description": "An int CPUtensor of size 1 specifying the tag to send the tensor with. This overrides the 'tag' argument of the op.",
          "name": "tag"
        }
      ],
      "outputs": [
        {
          "description": "The received tensor.",
          "name": "Y"
        },
        {
          "description": "The sender that sent the message as a CPUTensor of size 1 and of type int.",
          "name": "src"
        },
        {
          "description": "The tag that the message is sent with as a CPUTensor of size 1 and of type int.",
          "name": "tag"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SquaredL2DistanceGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Swish",
    "schema": {
      "description": "\nSwish takes one input data (Tensor) and produces one output data\n(Tensor) where the swish function, y = x / (1 + exp(-x)), is applied to the\ntensor elementwise.\n",
      "inputs": [
        {
          "description": "1D input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "1D output tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BatchGatherGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LearningRate",
    "schema": {
      "attributes": [
        {
          "description": "(float, required) base learning rate",
          "name": "base_lr",
          "option": "optional"
        },
        {
          "description": "(float, default 1.0) strategy for gamma enforcement",
          "name": "policy",
          "option": "optional"
        },
        {
          "description": "(float, default 1.0) used only for inv policy type",
          "name": "power",
          "option": "optional"
        },
        {
          "description": "(float, default 1.0) momentum of change",
          "name": "gamma",
          "option": "optional"
        },
        {
          "description": "(float, default 1.0) sampling rate on iterations",
          "name": "stepsize",
          "option": "optional"
        },
        {
          "description": "(boolean, default True) in alter policy",
          "name": "active_first",
          "option": "optional"
        },
        {
          "description": "(int64_t, required) in alter policy",
          "name": "active_period",
          "option": "optional"
        },
        {
          "description": "(int64_t, required) in alter policy",
          "name": "inactive_period",
          "option": "optional"
        },
        {
          "description": "(int, default -1) maximum iterations in this training run",
          "name": "max_iter",
          "option": "optional"
        },
        {
          "description": "(int, default 0) number of iterations over which to warmup lr",
          "name": "num_iter",
          "option": "optional"
        },
        {
          "description": "(float, default 0) starting multiplier for learning rate",
          "name": "start_multiplier",
          "option": "optional"
        },
        {
          "description": "(float, default 0) end multiplier for learning rate",
          "name": "end_multiplier",
          "option": "optional"
        },
        {
          "description": "(float, default 0.5) constant multiplier for learning rate",
          "name": "multiplier",
          "option": "optional"
        },
        {
          "description": "(float, default 1) start multiplier for learning rate",
          "name": "multiplier_1",
          "option": "optional"
        },
        {
          "description": "(float, default 1) end multiplier for learning rate",
          "name": "multiplier_2",
          "option": "optional"
        },
        {
          "description": "(int array, default empty) number of iterations for each sub learning rate policy in composite policy",
          "name": "sub_policy_num_iters",
          "option": "optional"
        },
        {
          "description": "",
          "name": "m1",
          "option": "optional"
        },
        {
          "description": "",
          "name": "n1",
          "option": "optional"
        },
        {
          "description": "",
          "name": "m2",
          "option": "optional"
        },
        {
          "description": "",
          "name": "n2",
          "option": "optional"
        },
        {
          "description": "",
          "name": "m3",
          "option": "optional"
        },
        {
          "description": "(float, default 0.005) max learning rate",
          "name": "max_lr",
          "option": "optional"
        },
        {
          "description": "defaults to 0.1",
          "name": "start_warmup_multiplier",
          "option": "optional"
        },
        {
          "description": "defaults to 10000000",
          "name": "constant_warmup_num_iter",
          "option": "optional"
        },
        {
          "description": "defaults to 10000000",
          "name": "linear_warmup_num_iter",
          "option": "optional"
        },
        {
          "description": "defaults to 0.05, part of CompositeCyclicalLRPolicy",
          "name": "cyclical_max_lr",
          "option": "optional"
        },
        {
          "description": "defaults to 1000000, part of CompositeCyclicalLRPolicy",
          "name": "cyclical_step_size",
          "option": "optional"
        },
        {
          "description": "defaults to 0.999, part of CompositeCyclicalLRPolicy",
          "name": "cyclical_decay",
          "option": "optional"
        },
        {
          "description": "defaults to 0.01, part of CompositeCosineLRPolicy",
          "name": "cosine_min_lr",
          "option": "optional"
        },
        {
          "description": "defaults to 0.05, part of CompositeCosineLRPolicy",
          "name": "cosine_max_lr",
          "option": "optional"
        },
        {
          "description": "defaults to 50, part of CompositeCosineLRPolicy",
          "name": "cosine_period",
          "option": "optional"
        },
        {
          "description": "defaults to 1,0, part of CompositeCosineLRPolicy",
          "name": "cosine_t_mult",
          "option": "optional"
        },
        {
          "description": "defaults to 0.99, part of CompositeCosineLRPolicy",
          "name": "cosine_lr_shrink",
          "option": "optional"
        }
      ],
      "description": "\nLearning rate is a decreasing function of time. With low learning rates the\nimprovements will be linear. With high learning rates they will start to look\nmore exponential. Learning rate is controlled by the following arguments:\n\n\nRequired:\n `iterations`\n `base_lr`: base learning rate\n `policy`: this controls how the learning rate is applied, options are:\n   `fixed`\n   `step`: uses `stepsize`, `gamma`\n   `exp`: uses `gamma`\n   `gate`: uses 'multiplier_1', 'multiplier_2', `num_iter``\n   `inv`: uses `gamma`, `power`\n   `linearWarmup`: uses `start_multiplier`, `num_iter`\n   `constantWarmup`: uses `multiplier`, `num_iter`\n   `alter`: uses  `active_first`, `active_period`, `inactive_period`\n   `hill`: uses those in both `linearWarmup` and `inv`, plus `end_multiplier`\n   `composite`: uses `sub_policy_num_iters` and additional args with format\n   `cyclic`: uses `max_lr`, `stepsize`\n   `cosine`: uses `min_lr`, `max_lr`, `period`, `t_mult`, `lr_shrink`\n   `constantThenLinearWarmup`: uses `start_warmup_multiplier`, `constant_warmup_num_iter`, `linear_warmup_num_iter`\n   `compositeCyclical`: uses `start_warmup_multiplier`, `constant_warmup_num_iter`, `linear_warmup_num_iter`, `cyclical_max_lr`, `cyclical_step_size`, `cyclical_decay`\n   `compositeCosine`: uses `start_warmup_multiplier`, `constant_warmup_num_iter`, `linear_warmup_num_iter`, `cosine_max_lr`, `cosine_period`, `cosine_t_mult`, `cosine_lr_shrink`\n   sub_policy_{sub_policy_index}_{sub_policy_arg}, for example:\n   sub_policy_0_policy: \"exp\", sub_policy_0_gamma: 0.99,\n   sub_policy_0_lr_scale: 1.2\n   sub_policy_0_policy: \"fixed\", sub_policy_0_lr_scale: 1.0\n   sub_policy_num_iters: [1000, 1000]\n\nOptional:\n  `stepsize`: defaults to 0\n  `max_lr`: defaults to 0.005\n  `gamma`: defaults to 0\n  `power`: defaults to 0\n  `num_iter`: defaults to 0\n  `start_multiplier`: defaults to 0\n  `multiplier`: defaults to 0.5\n  `multiplier_1`: defaults to 1\n  `multiplier_2`: defaults to 1\n  `m1`: defaults to 0.5, the first piece lr of piece warmup\n  `n1`: defaults to 0, iter threshold of the first piece lr\n  `m2`: defaults to 0.5, the second piece lr of piece warmup\n  `n2`: defaults to 0, iter threshold of the second piece lr\n  `m3`: defaults to 0.5, the third piece lr of piece warmup\n  `start_warmup_multiplier`: defaults to 0.1, part of constantThenLinearWarmup\n  `constant_warmup_num_iter`: defaults to 10000000, part of constantThenLinearWarmup and constantThenLinearWarmup\n  `linear_warmup_num_iter`: defaults to 10000000, part of constantThenLinearWarmup, CompositeCyclicalLRPolicy, CompositeCosineLRPolicy\n  `cyclical_max_lr`: defaults to 0.05, part of CompositeCyclicalLRPolicy\n  `cyclical_step_size`: defaults to 1000000, part of CompositeCyclicalLRPolicy\n  `cyclical_decay`: defaults to 1.0, part of CompositeCyclicalLRPolicy\n  `cosine_min_lr`:defaults to 0.01, part of CompositeCosineLRPolicy\n  `cosine_max_lr`:defaults to 0.05, part of CompositeCosineLRPolicy\n  `cosine_period`:defaults to 50, part of CompositeCosineLRPolicy\n  `cosine_t_mult`:defaults to 1.0, part of CompositeCosineLRPolicy\n  `cosine_lr_shrink`:defaults to 0.99, part of CompositeCosineLRPolicy\n\nUsage:\n  train_net.LearningRate(*iterations*, \"*label*\", base_lr=*float*,\n                         policy=\"policy_name\", stepsize=*int*, gamma=*float*)\n\n\nExample usage:\n  train_net.LearningRate(200, \"LR\", base_lr=-0.1,\n                         policy=\"step\", stepsize=20, gamma=0.9)\n",
      "inputs": [
        {
          "description": "description needed",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "description needed",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PReluGradient",
    "schema": {
      "description": "\n\nPReluGradient takes both Y and dY and uses this to update dX and dW according\nto the chain rule and derivatives of the rectified linear function.\n\n",
      "support_level": "default"
    }
  },
  {
    "name": "LengthsMean",
    "schema": {
      "description": "\nApplies 'Mean' to each segment of the input tensor. Segments are defined\nby their *LENGTHS*. *LENGTHS* is a vector that maps each of the slices of\n*DATA* to a particular segment. Values belonging to the same segment are\naggregated together and considered for the 'Mean' operation.\n\nFor example *LENGTHS = [2, 1]* stands for segments *DATA[0..1]* and *DATA[2]*\n\nThe sum of elements in *LENGTHS* must equal the number of elements in the first\ndimension of *DATA*. The length of *OUTPUT* is equal to the number of input\nsegments, i.e. len(*LENGTHS*).\n\nMean computes the element-wise mean of the input slices. Operation doesn't change the shape of the individual blocks.\n\n\nThe *LengthsMean* op takes two inputs *DATA* and *LENGTHS*, and produces a single output *OUTPUT*. The op finds the mean value in each of the segments of *DATA*, where segments are defined by their lengths.\nFor example, if $DATA = [2,4,3,1,2,10]$ and $LENGTHS = [2,3,1]$ then $OUTPUT = [mean([2,4]), mean([3,1,2]), mean([10])] = [3,2,10]$.\n\nGithub Link:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/segment_reduction_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LengthsMean\",\n    [\"DATA\", \"LENGTHS\"],\n    [\"OUTPUT\"],\n)\n\nworkspace.FeedBlob(\"DATA\", np.array([2,4,3,1,2,10]).astype(np.float32))\nprint(\"DATA:\\n\", workspace.FetchBlob(\"DATA\"))\n\nworkspace.FeedBlob(\"LENGTHS\", np.array([2,3,1]).astype(np.int32))\nprint(\"LENGTHS:\\n\", workspace.FetchBlob(\"LENGTHS\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"OUTPUT: \\n\", workspace.FetchBlob(\"OUTPUT\"))\n\n```\n\n**Result**\n\n```\n\nDATA:\n [ 2.  4.  3.  1.  2. 10.]\nLENGTHS:\n [2 3 1]\nOUTPUT:\n [ 3.  2. 10.]\n\n```\n\n</details>\n\n\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of len(LENGTHS) ",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Conv3DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CheckCounterDone",
    "schema": {
      "description": "\nIf the internal count value <= 0, outputs true, otherwise outputs false.\n\n  Github Links:\n  - https://github.com/pytorch/pytorch/blob/master/caffe2/operators/counter_ops.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\ncreatecounter_op = core.CreateOperator(\n    \"CreateCounter\",\n    [],\n    [\"counter\"],\n    init_count=5\n)\n\nretrievecount_op = core.CreateOperator(\n    \"RetrieveCount\",\n    [\"counter\"],\n    [\"count\"]\n)\n\ncheckcounterdone_op = core.CreateOperator(\n    \"CheckCounterDone\",\n    [\"counter\"],\n    [\"done\"]\n)\n\ncountup_op = core.CreateOperator(\n    \"CountUp\",\n    [\"counter\"],\n    [\"previous_count\"],\n)\n\ncountdown_op = core.CreateOperator(\n    \"CountDown\",\n    [\"counter\"],\n    [\"done\"],\n)\n\nresetcounter_op = core.CreateOperator(\n    \"ResetCounter\",\n    [\"counter\"],\n    [\"previous_count\"],\n    init_count=3\n)\n\n\n// Create counter\nworkspace.RunOperatorOnce(createcounter_op)\nprint(\"'counter' pointer:\", workspace.FetchBlob(\"counter\"))\n\n\n// Retrieve initial counter value\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"Initial 'count':\", workspace.FetchBlob(\"count\"))\n\n\n// Check if counter is done\nworkspace.RunOperatorOnce(checkcounterdone_op)\nprint(\"Initial 'done' value:\", workspace.FetchBlob(\"done\"))\n\n\n// Test CountUp operator\nprint(\"\\nTesting CountUp operator...\")\nfor i in range(5):\n    workspace.RunOperatorOnce(countup_op)\n    print(\"'previous_count' after CountUp:\", workspace.FetchBlob(\"previous_count\"))\n\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"'count' value after CountUp test:\", workspace.FetchBlob(\"count\"))\n\n\n// Test CountDown operator\nprint(\"\\nTesting CountDown operator...\")\nfor i in range(11):\n    workspace.RunOperatorOnce(countdown_op)\n    workspace.RunOperatorOnce(retrievecount_op)\n    print(\"'count' value after CountDown: {}\\t'done' value: {}\".format(workspace.FetchBlob(\"count\"), workspace.FetchBlob(\"done\")))\n```\n\n**Result**\n\n```\n'counter' pointer: counter, a C++ native class of type std::__1::unique_ptr<caffe2::Counter<long long>, std::__1::default_delete<caffe2::Counter<long long> > >.\nInitial 'count': 5\nInitial 'done' value: False\n\nTesting CountUp operator...\n'previous_count' after CountUp: 5\n'previous_count' after CountUp: 6\n'previous_count' after CountUp: 7\n'previous_count' after CountUp: 8\n'previous_count' after CountUp: 9\n'count' value after CountUp test: 10\n\nTesting CountDown operator...\n'count' value after CountDown: 9        'done' value: False\n'count' value after CountDown: 8        'done' value: False\n'count' value after CountDown: 7        'done' value: False\n'count' value after CountDown: 6        'done' value: False\n'count' value after CountDown: 5        'done' value: False\n'count' value after CountDown: 4        'done' value: False\n'count' value after CountDown: 3        'done' value: False\n'count' value after CountDown: 2        'done' value: False\n'count' value after CountDown: 1        'done' value: False\n'count' value after CountDown: 0        'done' value: False\n'count' value after CountDown: -1        'done' value: True\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<ptr>`)* A blob pointing to an instance of a counter.",
          "name": "counter"
        }
      ],
      "outputs": [
        {
          "description": "*(type: bool)* True if the internal count is zero or negative, otherwise False.",
          "name": "done"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LabelCrossEntropyGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Adam",
    "schema": {
      "attributes": [
        {
          "description": "Default 0.9",
          "name": "beta1",
          "option": "optional"
        },
        {
          "description": "Default 0.999",
          "name": "beta2",
          "option": "optional"
        },
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\nComputes the Adam update (https://arxiv.org/abs/1412.6980) for an\ninput gradient and momentum parameters. Concretely, given inputs\n(param, m1, m2, grad, lr, iters),\n\n    t = iters + 1\n    correction_multiplier = sqrt(1 - power(beta2, t)) /\n      (1 - power(beta1, t))\n    m1_o = (beta1 * m1) + (1 - beta1) * grad\n    m2_o = (beta2 * m2) + (1 - beta2) * np.square(grad)\n    grad_o = correction_multiplier * m1_o / \\\n        (sqrt(m2_o) + epsilon)\n    param_o = param + lr * grad_o\n\nand returns (param_o, m1_o, m2_o, grad_o), in which grad_o is an optional output\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "First moment history",
          "name": "moment_1"
        },
        {
          "description": "Second moment history",
          "name": "moment_2"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        },
        {
          "description": "iteration number",
          "name": "iter"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated first moment",
          "name": "output_moment_1"
        },
        {
          "description": "Updated second moment",
          "name": "output_moment_2"
        },
        {
          "description": "Optional Effective gradient",
          "name": "output_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FCGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "IsNaN",
    "schema": {
      "description": "Returns a new tensor with boolean elements representing if each element is NaN or not.",
      "inputs": [
        {
          "description": "Tensor to check for nan",
          "name": "tensor"
        }
      ],
      "outputs": [
        {
          "description": "Tensor containing a 1 at each location of NaN elements.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsWeightedSumWithMainInputGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ReduceBackSum",
    "schema": {
      "attributes": [
        {
          "description": "(*int*): number of dimensions to reduce (default=1)",
          "name": "num_reduce_dims",
          "option": "optional"
        }
      ],
      "description": "\nReduces the input tensor along the last dimension of the by applying **sum**.\n\nCan reduce more than one of the \"last\" dimensions by setting `num_reduce_dim`.\n\nA second (optional) input, `lengths`, can be passed, which enforces that only a subset of the elements are considered in the sum operation.\n- If input tensor `X` has shape $(d_0, d_1, d_2, ..., d_n)$, `lengths` must have shape $(d_0 * d_1 * d_2 * ... * d_{n-1})$.\n- The values of the `lengths` tensor determine how many of the values to consider for each vector in the $d_{n-1}$ dimension.\n\nFor example if $X = [[1,5,2,9],[4,1,8,2],[2,7,0,3]]$ and $lengths = [2,3,1]$, then $Y = [sum(1,5), sum(4,1,8), sum(2)] = [6, 13, 2]$\n\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduce_front_back_sum_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ReduceBackSum\",\n    [\"X\"],\n    [\"Y\"],\n    num_reduce_dim=2\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(1,2,3,3)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[[[2. 7. 7.]\n   [1. 1. 0.]\n   [9. 7. 2.]]\n\n  [[6. 6. 4.]\n   [1. 2. 6.]\n   [6. 6. 3.]]]]\nY: [[36. 40.]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor",
          "name": "X"
        },
        {
          "description": "(*Tensor`<int>`*): number of elements in each sample",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): reduced tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LogitGradient",
    "schema": {
      "attributes": [
        {
          "description": "small positive epsilon value, the default is 1e-6.",
          "name": "eps",
          "option": "optional"
        }
      ],
      "description": null,
      "inputs": [
        {
          "description": "input float tensor",
          "name": "X"
        },
        {
          "description": "input float tensor",
          "name": "dY"
        }
      ],
      "outputs": [
        {
          "description": "output float tensor",
          "name": "dX"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RowwiseMax",
    "schema": {
      "description": "\nCompute row-wise max reduction of the input tensor. This op takes one input, $X$, of shape $BxMxN$, where $B$ is the batch size, $M$ is number of rows, and $N$ is number of columns. The output of this op, $Y$, is a matrix of shape $BxM$, with one row for each element of the batch, and the same number of columns as the number of rows of the input tensor.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduction_ops.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduction_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"RowwiseMax\",\n    [\"X\"],\n    [\"Y\"]\n)\n\n// Create X, simulating a batch of 2, 4x4 matricies\nX = np.random.randint(0,high=20,size=(2,4,4))\nprint(\"X:\\n\",X)\n\n// Feed X into workspace\nworkspace.FeedBlob(\"X\", X.astype(np.float32))\n\n// Run op\nworkspace.RunOperatorOnce(op)\n\n// Collect Output\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[[ 5 12 10  1]\n  [ 4 16  2 15]\n  [ 5 11 12 15]\n  [15  4 17 19]]\n\n [[16  5  5 13]\n  [17  2  1 17]\n  [18  3 19  5]\n  [14 16 10 16]]]\nY:\n [[12. 16. 15. 19.]\n [16. 17. 19. 16.]]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "A tensor of dimensions $B x M x N$ to compute rowwise-max. Here, $B$ is batch size, and $M$ and $N$ are the number of rows and columns of each element of the batch, respectively.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "The output tensor of shape $B x M$, where each row represents the row-wise maximums for that element of the input batch.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Fail",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentMean",
    "schema": {
      "description": "\nApplies 'Mean' to each segment of input tensor. Segments need to be sorted and\ncontiguous. See also UnsortedSegmentMean that doesn't have this requirement.\n\nSEGMENT_IDS is a vector that maps each of the first dimension slices of the\nDATA to a particular group (segment). Values belonging to the same segment are\naggregated together.\n\nThe first dimension of the output is equal to the number of input segments,\ni.e. `SEGMENT_IDS[-1]+1`. Other dimensions are inherited from the input tensor.\n\nMean computes the element-wise mean of the input slices. Operation doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Vector with the same length as the first dimension of DATA and values in the range 0..K-1 and in increasing order that maps each slice of DATA to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of K (the number of segments).",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceFrontMaxGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "RemoveDataBlocks",
    "schema": {
      "description": "\nShrink the data tensor by removing data blocks with given zero-based indices in\nthe outermost dimension of the tensor. Indices are not assumed in any order or\nunique but with the range [0, blocks_size). Indices could be empty.\n  ",
      "inputs": [
        {
          "description": "a N-D data tensor, N >= 1",
          "name": "data"
        },
        {
          "description": "zero-based indices of blocks to be removed",
          "name": "indices"
        }
      ],
      "outputs": [
        {
          "description": "data after removing data blocks indexed by 'indices'",
          "name": "shrunk data"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SwapBestPath",
    "schema": {
      "description": "\nGiven a sequence of indices and a matrix, enforce that these indices have the\nbest columnwise scores\nscore\n",
      "inputs": [
        {
          "description": "N*D predictions matrix",
          "name": "predictions"
        },
        {
          "description": "N*1 vector holds the best path indices ",
          "name": "bestPath"
        }
      ],
      "outputs": [
        {
          "description": "N*D updated predictions matrix",
          "name": "new_predictions"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsIndicesInGradientWeightedSumWithMainInputGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "StringEndsWith",
    "schema": {
      "attributes": [
        {
          "description": "The suffix to check input strings against.",
          "name": "suffix",
          "option": "optional"
        }
      ],
      "description": "\nPerforms the ends-with check on each string in the input tensor.\nReturns tensor of boolean of the same dimension of input.\n",
      "inputs": [
        {
          "description": "Tensor of std::string.",
          "name": "strings"
        }
      ],
      "outputs": [
        {
          "description": "Tensor of bools of same shape as input.",
          "name": "bools"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "While",
    "schema": {
      "attributes": [
        {
          "description": "Net executed on each iteration",
          "name": "loop_net",
          "option": "optional"
        },
        {
          "description": "Net to (re)compute condition value",
          "name": "cond_net",
          "option": "optional"
        }
      ],
      "description": "\n'While' control operator, first input is a scalar boolean blob that stores loop's\ncondition value. Accepts 'loop_net' (required) and 'cond_net' (optional) arguments for\nloop's body and condition subnets respectively. If condition subnet is specified,\nit is executed before the first and after each iteration. Subnets are executed in\nthe same workspace as 'While'.\n    ",
      "inputs": [
        {
          "description": "Scalar boolean condition",
          "name": "condition"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Range",
    "schema": {
      "description": "\nGenerates an output tensor within the half-open interval $[start, stop)$ (the interval including start but excluding stop).\n- The `start` input is optional, and defaults to 0 when not set.\n- The `step` input is optional, and defaults to 1 when not set.\n- The type of the `output` tensor is determined by the types of inputs used.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/utility_ops.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/utility_ops.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Range\",\n    [\"start\", \"stop\", \"step\"],\n    [\"output\"]\n)\n\nworkspace.FeedBlob(\"start\", np.array(4, dtype=np.int32))\nworkspace.FeedBlob(\"stop\", np.array(17, dtype=np.int32))\nworkspace.FeedBlob(\"step\", np.array(2, dtype=np.int32))\nprint(\"start:\", workspace.FetchBlob(\"start\"))\nprint(\"stop:\", workspace.FetchBlob(\"stop\"))\nprint(\"step:\", workspace.FetchBlob(\"step\"))\nworkspace.RunOperatorOnce(op)\nprint(\"output:\", workspace.FetchBlob(\"output\"))\n\n```\n\n**Result**\n\n```\n\nstart: 4\nstop: 17\nstep: 2\noutput: [ 4  6  8 10 12 14 16]\n\n```\n\n</details>\n        ",
      "inputs": [
        {
          "description": "(*Tensor*): [OPTIONAL] scalar or 1-element tensor containing the start of the interval (inclusive) (default=0)",
          "name": "start"
        },
        {
          "description": "(*Tensor*): scalar or 1-element tensor containing the end of the interval (exclusive)",
          "name": "stop"
        },
        {
          "description": "(*Tensor*): [OPTIONAL] scalar or 1-element tensor specifying the spacing between values (default=1)",
          "name": "step"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor*): 1D tensor of same type as inputs that contains the sequence",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8AddRelu",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "\n    Performs element-wise binary Add (with no broadcast support). \"\n    \"Output will go through rectified linear \"\n    \"function, where y = max(0, x).\n",
      "inputs": [
        {
          "description": "First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "Second operand. It should be of the same size as A.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "Result, has same dimensions and type as A",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SquareRootDivide",
    "schema": {
      "description": "\nGiven DATA tensor with first dimension N and SCALE vector of the same size N\nproduces an output tensor with same dimensions as DATA. Which consists of DATA\nslices. i-th slice is divided by sqrt(SCALE[i]) elementwise. If SCALE[i] == 0\noutput slice is identical to the input one (no scaling)\n\nExample:\n\n  Data = [\n    [2.0, 4.0],\n    [9.0, 12.0]\n  ]\n\n  SCALE = [4, 9]\n\n  OUTPUT = [\n    [1.0, 2.0],\n    [3.0, 4.0]\n  ]\n\n",
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Sign",
    "schema": {
      "description": "\nComputes sign for each element of the input: -1, 0 or 1.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n\"Sign\",\n[\"X\"],\n[\"Y\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.rand(3, 3).astype(np.float32) - np.random.rand(3, 3).astype(np.float32)))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[ 0.02816287  0.22408086 -0.30342305]\n[-0.18481976  0.03948995  0.39698976]\n[-0.63304734 -0.6919183  -0.31524038]]\nY:\n[[ 1.  1. -1.]\n[-1.  1.  1.]\n[-1. -1. -1.]]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Xor",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise logical operation **xor** (with limited broadcast support).\nBoth input operands should be of type `bool`.\n\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Xor\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", (np.random.rand(3, 3) > 0.5))\nworkspace.FeedBlob(\"B\", (np.random.rand(3, 3) > 0.5))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\n\nA:\n[[ True  True  True]\n [False False  True]\n [False  True False]]\nB:\n[[False False False]\n [ True  True  True]\n [False False False]]\nC:\n[[ True  True  True]\n [ True  True False]\n [False  True False]]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "*(type: Tensor`<bool>`)* First operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<bool>`)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<bool>`)* Output tensor of booleans. Has same dimensions as input `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SigmoidGradient",
    "schema": {
      "description": "\nSigmoidGradient takes both Y and dY and uses this to update dX according to the\nchain rule and derivatives of the sigmoid function.\n",
      "support_level": "default"
    }
  },
  {
    "name": "MergeMultiScalarFeatureTensorsGradient",
    "schema": {
      "description": "Explode given multi-feature tensors with scalar features into many.\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": ".lengths",
          "name": "in1_lengths"
        },
        {
          "description": ".values_grad",
          "name": "out_values_grad"
        }
      ],
      "outputs": [
        {
          "description": ".values_grad",
          "name": "in1_values_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Sin",
    "schema": {
      "description": "\nCalculates the sine of the given input tensor, element-wise.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/sin_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Sin\",\n    [\"X\"],\n    [\"Y\"]\n)\n\nworkspace.FeedBlob(\"X\", np.random.rand(5).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX: [0.8466114  0.1803606  0.5601509  0.04959291 0.64770824]\nY: [0.74903965 0.17938434 0.5313141  0.04957259 0.60336035]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor calculated as the sine of the input tensor, element-wise.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Log",
    "schema": {
      "description": "\nCalculates the natural log of the given input tensor ($ln(x)$), element-wise. This\noperation can be done in an in-place fashion too, by providing the same input\nand output blobs.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/log_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Log\",\n    [\"X\"],\n    [\"X\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.rand(3,3)).astype(np.float32))\nprint(\"X before running op:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"X after running op:\", workspace.FetchBlob(\"X\"))\n\n```\n\n**Result**\n\n```\n\nX before running op:\n[[0.07341351 0.15404125 0.386613  ]\n [0.34090295 0.99727786 0.24141751]\n [0.32016268 0.8724168  0.93515724]]\nX after running op:\n[[-2.6116474  -1.8705349  -0.9503311 ]\n [-1.0761575  -0.00272586 -1.4212275 ]\n [-1.138926   -0.13648799 -0.06704059]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor computed as the natural log of the input tensor computed, element-wise.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "DenseVectorToIdList",
    "schema": {
      "description": "\nDenseVectorToIdList: Convert a blob with dense feature into a ID_LIST.\n\nAn ID_LIST is a list of IDs (may be ints, often longs) that represents a single\nfeature. As described in https://caffe2.ai/docs/sparse-operations.html, a batch\nof ID_LIST examples is represented as a pair of lengths and values where the\n`lengths` (int32) segment the `values` or ids (int32/int64) into examples.\n\nInput is a single blob where the first dimension is the batch size and the\nsecond dimension is the length of dense vectors. This operator produces a\nID_LIST where out_values are the indices of non-zero entries\nand out_lengths are the number of non-zeros entries in each row.\n\n",
      "inputs": [
        {
          "description": "A data blob of dense vectors",
          "name": "values"
        }
      ],
      "outputs": [
        {
          "description": "Lengths of the sparse feature",
          "name": "out_lengths"
        },
        {
          "description": "Values of the sparse feature",
          "name": "out_values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Reshape",
    "schema": {
      "attributes": [
        {
          "description": "New shape. Do not set if using `new_shape` input.",
          "name": "shape",
          "option": "optional"
        }
      ],
      "description": "\nReshape the input tensor similar to numpy's\n[reshape](https://docs.scipy.org/doc/numpy/reference/generated/numpy.reshape.html).\n\nTakes a tensor as input and an optional tensor specifying the new shape. When\nthe second input is absent, an extra argument shape must be specified. Outputs\nthe reshaped tensor as well as the original shape.\n\nAt most one dimension of the new shape can be -1. In this case, the value is\ninferred from the size of the tensor and the remaining dimensions. A dimension\ncould also be 0, in which case the actual dimension value is going to be copied\nfrom the input tensor.\n\nFor empty tensor, we will set the -1 dimension to be 0 (if one dimension is -1).\nWhen the tensor is empty, dimension of 0 will remain to be 0.\nE.g: data=np.empty(shape=[4, 0]), shape=[0, -1], the output tensor will be\nnp.emtpy(shape=[0, 0])\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reshape_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Reshape\",\n    [\"data\"],\n    [\"reshaped\", \"old_shape\"],\n    shape=(3,2)\n)\n\nworkspace.FeedBlob(\"data\", (np.random.randint(100, size=(6))))\nprint(\"data:\", workspace.FetchBlob(\"data\"))\nworkspace.RunOperatorOnce(op)\nprint(\"reshaped:\", workspace.FetchBlob(\"reshaped\"))\nprint(\"old_shape:\", workspace.FetchBlob(\"old_shape\"))\n```\n\n**Result**\n\n```\ndata: [86 60 85 96  7 37]\nreshaped: [[86 60]\n          [85 96]\n          [ 7 37]]\nold_shape: [6]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor)* Input tensor.",
          "name": "data"
        },
        {
          "description": "*(type: Tensor`<int>`)* [OPTIONAL] Tensor containing new shape.",
          "name": "new_shape"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor)* Reshaped output tensor.",
          "name": "reshaped"
        },
        {
          "description": "*(type: Tensor`<int>`)* Tensor containing old shape of `data`.",
          "name": "old_shape"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AveragePoolGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "EnqueueBlobs",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "DepthConcat",
    "schema": {
      "description": "Backward compatible operator name for Concat.",
      "support_level": "default"
    }
  },
  {
    "name": "GatherPadding",
    "schema": {
      "attributes": [
        {
          "description": "Outer-size of padding present around each range.",
          "name": "padding_width",
          "option": "optional"
        },
        {
          "description": "(Optional) Specifies a different end-padding width.",
          "name": "end_padding_width",
          "option": "optional"
        }
      ],
      "description": "\nGather the sum of start and end paddings in a padded input sequence. Used in\norder to compute the gradients of AddPadding w.r.t the padding tensors.\n",
      "inputs": [
        {
          "description": "T<N, D1..., Dn> Padded input data",
          "name": "data_in"
        },
        {
          "description": "(i64) Num of elements in each range. sum(lengths) = N. If not provided, considers all data as a single segment.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Sum of all start paddings, or of all paddings if end_padding_sum is not provided.",
          "name": "padding_sum"
        },
        {
          "description": "T<D1..., Dn> Sum of all end paddings, if provided.",
          "name": "end_padding_sum"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "DepthSplit",
    "schema": {
      "description": "Backward compatible operator name for Split.",
      "support_level": "default"
    }
  },
  {
    "name": "UnsortedSegmentMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Conv1DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LengthsSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ReduceMin",
    "schema": {
      "attributes": [
        {
          "description": "A list of integers, along which to reduce.",
          "name": "axes",
          "option": "optional"
        },
        {
          "description": "Keep the reduced dimension(s) or not, default True keeps the reduced dimension(s).",
          "name": "keepdims",
          "option": "optional"
        }
      ],
      "description": "\n  Computes the min of the input tensor's element along the provided axes.\n  The resulted tensor has the same rank as the input if keepdims equal True.\n  If keepdims equal false, then the resulted tensor have the reduced dimension\n  pruned.\n",
      "inputs": [
        {
          "description": "An input tensor.",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "Reduced output tensor.",
          "name": "reduced"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StatRegistryExport",
    "schema": {
      "attributes": [
        {
          "description": "(default true) Whether to atomically reset the counters afterwards.",
          "name": "reset",
          "option": "optional"
        }
      ],
      "description": null,
      "inputs": [
        {
          "description": "If provided, export values from given StatRegistry.Otherwise, export values from the global singleton StatRegistry.",
          "name": "handle"
        }
      ],
      "outputs": [
        {
          "description": "1D string tensor with exported key names",
          "name": "keys"
        },
        {
          "description": "1D int64 tensor with exported values",
          "name": "values"
        },
        {
          "description": "The unix timestamp at counter retrieval.",
          "name": "timestamps"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Free",
    "schema": {
      "description": "\nFrees the content of the blobs. The input and output blobs should be\none-to-one inplace.",
      "support_level": "default"
    }
  },
  {
    "name": "FlexibleTopK",
    "schema": {
      "description": "\nGiven two tensors: X and K,\nretrieve the top K[..., 1] elements from X on the last dimension.\nX is an input tensor of shape [a_1, a_2, ..., a_n, r].\nK is an input tensor of shape [a_1, a_2, ..., a_n, 1],\nwhere for each element, r >= K[..., 1] > 0\nOutput two outputs:\n-Flatten values tensor of shape [ \\sum_i K[i, 1] ] which contains the values of\n the top K[..., 1]  elements along the last dimension\n-Flatten indices tensor of shape [ \\sum_i K[i, 1] ] which contains the indices\n of the top K[..., 1]  elements, flatten indices from the input tensor).\nThese two outputs should be used with the input K, so that we know which indices\nin X are picked.\n\nGiven two equivalent values, this operator uses the indices along the last dim-\nension as a tiebreaker. That is, the element with the lower index will appear\nfirst.\n    ",
      "inputs": [
        {
          "description": "Tensor of shape [a_1, a_2, ..., a_n, r]",
          "name": "X"
        },
        {
          "description": "Tensor of shape [a_1, a_2, ..., a_n, 1]",
          "name": "K"
        }
      ],
      "outputs": [
        {
          "description": "Tensor of shape [ \\sum_i K[i, 1] ] containing top K[..., 1] values from the input tensor",
          "name": "Flatten values"
        },
        {
          "description": "Tensor of shape [ \\sum_i K[i, 1] ] containing the indices into the flatten input",
          "name": "Flatten indices"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceL1",
    "schema": {
      "attributes": [
        {
          "description": "(*Tuple(int)*): list of axes to reduce",
          "name": "axes",
          "option": "optional"
        },
        {
          "description": "(*int*): set to 1 to keep the reduced dimension(s) (default=1), else set to 0 to not keep the reduced dimension(s)",
          "name": "keepdims",
          "option": "optional"
        }
      ],
      "description": "\nComputes the **L1 norm** of the input tensor's elements along the provided `axes`. The resulting tensor has the same rank as the input if the `keepdims` argument equals 1 (default). If `keepdims` is set to 0, then the `axes` dimensions are pruned.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduce_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ReduceL1\",\n    [\"X\"],\n    [\"Y\"],\n    axes=(0,1),\n    keepdims=0\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(1,2,5,5)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[[[ 2.  7.  6.  4.  5.]\n   [ 2.  1.  9.  8.  7.]\n   [ 4.  9.  1.  0.  0.]\n   [ 6.  4.  0.  8.  1.]\n   [ 1.  7.  1.  0.  2.]]\n\n  [[ 5.  8.  1.  7.  7.]\n   [ 4.  5.  6.  5.  4.]\n   [ 1.  9.  6.  6.  3.]\n   [ 6.  6.  8.  8.  4.]\n   [ 2.  3.  5.  8.  1.]]]]\n\nY:\n[[  7.  15.   7.  11.  12.]\n [  6.   6.  15.  13.  11.]\n [  5.  18.   7.   6.   3.]\n [ 12.  10.   8.  16.   5.]\n [  3.  10.   6.   8.   3.]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): reduced tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "NumpyTile",
    "schema": {
      "description": null,
      "inputs": [
        {
          "description": "The input tensor.",
          "name": "data"
        },
        {
          "description": "1-D Tensor specifying how many times to repeat each axis.",
          "name": "repeats"
        }
      ],
      "outputs": [
        {
          "description": "Tensor that will contain input replicated along the given axis.",
          "name": "tiled_data"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "EluGradient",
    "schema": {
      "description": "\nEluGradient takes both Y and dY and uses this to update dX according to the\nchain rule and derivatives of the rectified linear function.\n",
      "support_level": "default"
    }
  },
  {
    "name": "ArgMax",
    "schema": {
      "attributes": [
        {
          "default": -1,
          "description": "The axis to get argmax.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": true,
          "description": "If True (default), the output tensor shape will match the input tensor shape except the `axis` dimension equals 1. Else, the `axis` dimension of the output tensor is removed.",
          "name": "keepdims",
          "option": "optional",
          "type": "boolean"
        }
      ],
      "description": "\nRetrieve the argmax of an axis dimension specified by the `axis`\nargument. Given an input tensor and two arguments (`axis` and\n`keepdims`), returns a tensor containing the indices of the largest\nelement along the given axis. If the `keepdims` arg is *True* (default),\nthe shape of the output tensor matches the input tensor except the\n`axis` dimension equals 1. Else, the `axis` dimension of the output\ntensor is removed.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/arg_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ArgMax\",\n    [\"X\"],\n    [\"Indices\"],\n    axis=2,\n    keepdims=False\n)\n\nworkspace.FeedBlob(\"X\", (np.random.randint(10, size=(3,3,3))).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Indices:\", workspace.FetchBlob(\"Indices\"))\n\n```\n\n**Result**\n\n```\nX: [[[4. 9. 6.]\n  [6. 6. 1.]\n  [9. 5. 4.]]\n\n [[6. 7. 4.]\n  [7. 9. 1.]\n  [3. 2. 8.]]\n\n [[3. 4. 6.]\n  [5. 2. 7.]\n  [1. 5. 7.]]]\nIndices: [[1 0 0]\n [1 1 2]\n [2 2 2]]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Tensor of indices for the largest values.",
          "name": "Indices"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "IndexGet",
    "schema": {
      "description": "\nGiven an index handle and a tensor of keys, return an Int tensor of same shape\ncontaining the indices for each of the keys. If the index is frozen, unknown\nentries are given index 0. Otherwise, new entries are added into the index.\nIf an insert is necessary but max_elements has been reached, fail.\n",
      "inputs": [
        {
          "description": "Pointer to an Index instance.",
          "name": "handle"
        },
        {
          "description": "Tensor of keys to be looked up.",
          "name": "keys"
        }
      ],
      "outputs": [
        {
          "description": "Indices for each of the keys.",
          "name": "indices"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HeatmapMaxKeypoint",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "MomentsGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "L1Distance",
    "schema": {
      "description": "\nComputes the row-wise L1 Distance between the two input tensors $X$ and $Y$, which is defined as\n\n$$L1Distance(\\mathbf{x},\\mathbf{y}) = \\sum_{i}\\mid x_i - y_i\\mid$$\n\nNote, both inputs must either be 1-dimensional or 2-dimensional and both must have the same shape. The output $Z$ will be 1-dimensional regardless and its length will equal the number of rows in the inputs.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/distance_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/distance_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"L1Distance\",\n    [\"X\", \"Y\"],\n    [\"Z\"]\n)\n\n// Create X\nX = 5*np.ones((1, 4))\nprint(\"X:\\n\",X)\n\n// Create Y\nY = np.ones((1, 4))\nprint(\"Y:\\n\",Y)\n\n// Feed X & Y into workspace\nworkspace.FeedBlob(\"X\", X.astype(np.float32))\nworkspace.FeedBlob(\"Y\", Y.astype(np.float32))\n\n// Run op\nworkspace.RunOperatorOnce(op)\n\n// Collect Output\nprint(\"Z:\\n\", workspace.FetchBlob(\"Z\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[5. 5. 5. 5.]]\nY:\n [[1. 1. 1. 1.]]\nZ:\n [16.]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "First input tensor. (1D or 2D)",
          "name": "X"
        },
        {
          "description": "Second input tensor. (must have the same shape as $X$)",
          "name": "Y"
        }
      ],
      "outputs": [
        {
          "description": "1D output tensor. One value for each row of the inputs.",
          "name": "Z"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Allreduce",
    "schema": {
      "description": "\nDoes an allreduce operation among the nodes. Currently only Sum is supported.\n",
      "inputs": [
        {
          "description": "The common world.",
          "name": "comm_world"
        },
        {
          "description": "A tensor to be allreduced.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "The allreduced tensor, same on all nodes.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Onnxifi",
    "schema": {
      "attributes": [
        {
          "description": "(string default=\"\") Serialized ONNX model to be converted to backend representation",
          "name": "onnx_model",
          "option": "optional"
        },
        {
          "description": "Initialization pair indicating the mapping of the name between NetDef and ONNX model",
          "name": "initializers",
          "option": "optional"
        },
        {
          "description": "A list of key/value pairs indicating which input index to look up for real batch size for the given max output batch size",
          "name": "output_resize_hints",
          "option": "optional"
        }
      ],
      "description": "\n    The Onnxifi operator is a black-box operator to lower the computation to Onnxifi backend\n    ",
      "support_level": "default"
    }
  },
  {
    "name": "Partition",
    "schema": {
      "attributes": [
        {
          "description": "(int, default 0) If set, the operator transforms the first tensor values as floor(X_ij / num_partitions)",
          "name": "pack_first_input",
          "option": "optional"
        }
      ],
      "description": "\nSplits the input int tensor into multiple ones according to the first tensor.\n\nTakes the first input and partitions it to shards according to the remainder of\nvalues modulo the number of partitions. It requires that the first tensor is of\nintegral type. The number of partitions is derived as (num_output / num_input).\n\nIf additional inputs are present they must have the same shape as the first\ninput, optionally with extra trailing dimensions. They will be partitioned\naccordingly to the first input.\n\nOptional arg 'pack_first_input' transforms the first tensor values as\nX_ij / num_partitions.\n\nOutputs are ordered as\nX_0_part_0, X_1_part_0, ..., X_N-1_part_0, X_0_part_1, ..., X_N-1_part_K-1\n",
      "inputs": [
        {
          "description": "Input tensor containing data to be partitioned. The number of input tensors might be greater than 1 but must have the same shape as the previous tensors.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Output Partitions. The number of output tensors has to be a multiple of the number of input tensors.",
          "name": "partitions"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsTopKGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "FlexibleTopKGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ThrowException",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LengthsMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Tile",
    "schema": {
      "attributes": [
        {
          "description": "(*int*): number of replicas",
          "name": "tiles",
          "option": "optional"
        },
        {
          "description": "(*int*): axis to replicate along",
          "name": "axis",
          "option": "optional"
        }
      ],
      "description": "\nConstructs a tensor by tiling a given tensor along a specified axis. This operation creates a new tensor by replicating the input tensor a number of times specified by the `tiles` argument along the `axis` dimension. The output tensor's `axis` dimension has $(X.dims(axis) * tiles)$ elements.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/tile_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Tile\",\n    [\"X\", \"tiles\", \"axis\"],\n    [\"Y\"]\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(5,5)))\nworkspace.FeedBlob(\"tiles\", np.array([5]).astype(np.int32))\nworkspace.FeedBlob(\"axis\", np.array([1]).astype(np.int32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[9 1 7 1 3]\n [2 3 6 2 5]\n [0 9 2 6 4]\n [5 8 1 5 9]\n [2 0 1 3 7]]\nY:\n[[9 1 7 1 3 9 1 7 1 3 9 1 7 1 3 9 1 7 1 3 9 1 7 1 3]\n [2 3 6 2 5 2 3 6 2 5 2 3 6 2 5 2 3 6 2 5 2 3 6 2 5]\n [0 9 2 6 4 0 9 2 6 4 0 9 2 6 4 0 9 2 6 4 0 9 2 6 4]\n [5 8 1 5 9 5 8 1 5 9 5 8 1 5 9 5 8 1 5 9 5 8 1 5 9]\n [2 0 1 3 7 2 0 1 3 7 2 0 1 3 7 2 0 1 3 7 2 0 1 3 7]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor*): input tensor",
          "name": "X"
        },
        {
          "description": "(*Tensor`<int>`*): [OPTIONAL] number of replicas (overrides `tiles` argument)",
          "name": "tiles"
        },
        {
          "description": "(*Tensor`<int>`*): [OPTIONAL] axis to replicate along (overrides `axis` argument)",
          "name": "axis"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor*): output tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Asin",
    "schema": {
      "description": "\nCalculates the arcsine of the given input tensor, element-wise.\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The arcsine of the input tensor computed element-wise",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SplitByLengths",
    "schema": {
      "attributes": [
        {
          "description": "Which axis to split on",
          "name": "axis",
          "option": "optional"
        },
        {
          "description": "Either NHWC or NCWH, will split on C axis, defaults to NCHW",
          "name": "order",
          "option": "optional"
        }
      ],
      "description": "\nSplit a tensor into a list of tensors, given a lengths input, along the specified\n'axis'. If `K` outputs are provided, the op assumes `len(lengths) % K == 0`.\nThe `input` will be split into `K` parts. Each part of length\n`sum(lengths[i*k:i*k+k))`",
      "inputs": [
        {
          "description": "The tensor to split",
          "name": "input"
        },
        {
          "description": "The tensor `l_i` indicates the logic block of input.",
          "name": "legnths"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CreateBlobsQueueDB",
    "schema": {
      "attributes": [
        {
          "description": "(default: -1 (no key)) index of blob for DB key in the BlobsQueue.",
          "name": "key_blob_index",
          "option": "optional"
        },
        {
          "description": "(default: 0) index of blob for DB value in the BlobsQueue.",
          "name": "value_blob_index",
          "option": "optional"
        },
        {
          "description": "(default: 0.0 (no timeout)) Timeout in seconds for reading from the BlobsQueue.",
          "name": "timeout_secs",
          "option": "optional"
        }
      ],
      "description": "Create a DBReader from a BlobsQueue",
      "inputs": [
        {
          "description": "The shared pointer to a queue containing Blobs.",
          "name": "queue"
        }
      ],
      "outputs": [
        {
          "description": "The DBReader for the given BlobsQueue",
          "name": "reader"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GRUUnitGradient",
    "schema": {
      "attributes": [
        {
          "description": "When false, the sequence lengths input is left out, and all following inputs are shifted left by one.",
          "name": "sequence_lengths",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CloseBlobsQueue",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Int8Sigmoid",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "\nApply the Sigmoid function element-wise to the input tensor. This is often used\nas a non-linear activation function in a neural network. The sigmoid function is\ndefined as:\n\n$$Sigmoid(x) = \\frac{1}{1+\\exp(-x)}$$\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/sigmoid_op.cc\n",
      "inputs": [
        {
          "description": "The input tensor that's coerced into a 2D matrix of size (NxD) as described above.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The sigmoid normalized output values with the same shape as input tensor.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GRUUnit",
    "schema": {
      "attributes": [
        {
          "description": "Bool to determine if hidden state is zeroes or passed along for timesteps past the given sequence_length.",
          "name": "drop_states",
          "option": "optional"
        },
        {
          "description": "When false, the sequence lengths input is left out, and all following inputs are shifted left by one.",
          "name": "sequence_lengths",
          "option": "optional"
        }
      ],
      "description": "\nGRUUnit computes the activations of a standard GRU,\nin a sequence-length aware fashion.\n\nConcretely, given the (fused) inputs X (TxNxD), the previous hidden\nstate (NxD), and the sequence lengths (N), computes the GRU\nactivations, avoiding computation if the input is invalid (as in, the\nvalue at X[t][n] >= seqLengths[n].\n\n",
      "outputs": [
        {
          "description": "The new GRU hidden state calculated by this op.",
          "name": "hidden"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentSum",
    "schema": {
      "description": "\nApplies 'Sum' to each segment of input tensor. Segments need to be sorted and\ncontiguous. See also UnsortedSegmentSum that doesn't have this requirement.\n\nSEGMENT_IDS is a vector that maps each of the first dimension slices of the\nDATA to a particular group (segment). Values belonging to the same segment are\naggregated together.\n\nThe first dimension of the output is equal to the number of input segments,\ni.e. `SEGMENT_IDS[-1]+1`. Other dimensions are inherited from the input tensor.\n\nSummation is done element-wise across slices of the input tensor and doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Vector with the same length as the first dimension of DATA and values in the range 0..K-1 and in increasing order that maps each slice of DATA to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of K (the number of segments).",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Sqrt",
    "schema": {
      "description": "\nPerforms element-wise square-root ($\\sqrt{x}$) of input tensor $X$.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/sqrt_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Sqrt\",\n    [\"X\"],\n    [\"Y\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.randint(10, size=(3,3))).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[8. 3. 3.]\n [4. 0. 0.]\n [1. 2. 5.]]\nY:\n[[2.8284268  1.7320508  1.7320508 ]\n [1.9999999  0.         0.        ]\n [0.99999994 1.4142134  2.236068  ]]\n\n```\n\n</details>\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "TTLinearGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseFtrl",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LambdaRankNdcgGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsWeightedSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseNormalize",
    "schema": {
      "attributes": [
        {
          "description": "A bool variable to control whether to use max norm     or constant norm. When use_max_norm = false, constant norm is used so that     all the embedding vectors are scaled to have a L2 norm equals to A     (see blow argument norm=A). If use_max_norm = true,     max norm is used so that embedding is scaled so that its l2 norm is no larger     than A. If an embedding's norm is less than A originally,     the embedding is left unchanged.    The default is True.",
          "name": "use_max_norm",
          "option": "optional"
        },
        {
          "description": "L2 norm of the embedding. The default is 1.0.",
          "name": "norm",
          "option": "optional"
        }
      ],
      "description": "\nGiven a sparse matrix, apply max_norm or constant_norm sparse regularization.\n",
      "inputs": [
        {
          "description": "Parameters to be normalized",
          "name": "param"
        },
        {
          "description": "Sparse indices",
          "name": "indices"
        },
        {
          "description": "Gradient computed (optional - not used, this argument is for backwards compatibility)",
          "name": "grad"
        }
      ],
      "outputs": [
        {
          "description": "Normalized parameters",
          "name": "output_param"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LeakyRelu",
    "schema": {
      "attributes": [
        {
          "default": 0.01,
          "description": "Coefficient of leakage.",
          "name": "alpha",
          "option": "optional",
          "type": "float32"
        }
      ],
      "description": "\nThe *LeakyRelu* op takes one input tensor $X$ and an argument $alpha$, and produces one output tensor $Y$ of the same shape as $X.$ The op performs the element wise leaky relu operation, defined as\n\n$$y=LeakyRelu(x) =\\begin{cases}\\alpha x & x < 0\\\\x & otherwise\\end{cases}$$\n\nThe default value of *alpha* is 0.01.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/leaky_relu_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/leaky_relu_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LeakyRelu\",\n    [\"X\"],\n    [\"Y\"],\n    alpha=0.01\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(3, 3).astype(np.float32))\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\n\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[-0.91060215  0.09374836  2.1429708 ]\n [-0.748983    0.19164062 -1.5130422 ]\n [-0.29539835 -0.8530696   0.7673204 ]]\n\nY:\n [[-0.00910602  0.09374836  2.1429708 ]\n [-0.00748983  0.19164062 -0.01513042]\n [-0.00295398 -0.0085307   0.7673204 ]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "Input tensor of data to be operated on.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor, calculated as described above.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AddGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LeakyReluGradient",
    "schema": {
      "attributes": [
        {
          "description": "Coefficient of leakage",
          "name": "alpha",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ReadRandomBatch",
    "schema": {
      "attributes": [
        {
          "description": "Number of top-level entries to read.",
          "name": "batch_size",
          "option": "optional"
        },
        {
          "description": "(bool) Repeat the dataset indefinitely",
          "name": "loop_over",
          "option": "optional"
        }
      ],
      "description": "\nRead the next batch of examples out of the given cursor,\nidx blob, offset matrix and data blobs.\n\nInput(0) is a blob pointing to a TreeCursor,\nInput(1) is a blob pointing to the shuffled idx\nInput(2) is a blob pointing to the offset matrix and\n[Input(3),... Input(num_fields)] a list of tensors containing the data for\neach field of the dataset.\n\nReadRandomBatch is thread safe.\n",
      "inputs": [
        {
          "description": "A blob containing a pointer to the cursor.",
          "name": "cursor"
        },
        {
          "description": "idx with a shuffled order.",
          "name": "idx"
        },
        {
          "description": "offset matrix containing length offset info.",
          "name": "offsetsmat"
        },
        {
          "description": "First dataset field",
          "name": "dataset_field_0"
        }
      ],
      "outputs": [
        {
          "description": "Tensor containing the next batch for field 0.",
          "name": "field_0"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HalfToFloat",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ReduceScatter",
    "schema": {
      "description": "\nDoes reduce-scatter operation among the nodes. Currently only Sum is supported.\n",
      "inputs": [
        {
          "description": "The common world.",
          "name": "comm_world"
        },
        {
          "description": "A tensor to be reduce-scattered.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "The reduced tensor, scattered on all nodes.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SeluGradient",
    "schema": {
      "attributes": [
        {
          "description": "(float) default to 1.6732~; affects the activation function itself.This should go with the weight initialization in the paper.  See https://arxiv.org/abs/1706.02515",
          "name": "alpha",
          "option": "optional"
        },
        {
          "description": "(float) default to 1.0507~; affects the activation function itself.",
          "name": "scale",
          "option": "optional"
        }
      ],
      "description": "\nSeluGradient takes both Y and dY and uses this to update dX according to the\nchain rule and derivatives of the selu function.\n",
      "inputs": [
        {
          "description": "input tensor",
          "name": "Y"
        },
        {
          "description": "input tensor",
          "name": "dY"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsToShape",
    "schema": {
      "description": "\nThis operator takes a list of $N$ equal integers as input which represent the lengths of $N$ vectors. The output is the calculated shape of the matrix if the $N$ integers were combined into a single matrix.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/utility_ops.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/utility_ops.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LengthsToShape\",\n    [\"X\"],\n    [\"Y\"]\n)\n\n// Create X: Sample softmax output for 5-class model\nX = np.array([2,2,2,2,2,2,2,2,2,2])\nprint(\"X:\\n\",X)\n\n// Feed X into workspace\nworkspace.FeedBlob(\"X\", X.astype(np.int32))\n\n// Run op\nworkspace.RunOperatorOnce(op)\n\n// Collect Output\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [2 2 2 2 2 2 2 2 2 2]\nY:\n [10  2]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "List, of length $N$, of equal integers representing the lengths of several vectors.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Vector of length 2 describing the dimensions of the data if the $N$ vectors from the input were combined to a single matrix.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AveragePool1DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ResetCounter",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Resets counter to this value, must be >= 0.",
          "name": "init_count",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nResets a count-down counter with initial value specified by the `init_count`\nargument.\n\n  Github Links:\n  - https://github.com/pytorch/pytorch/blob/master/caffe2/operators/counter_ops.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\ncreatecounter_op = core.CreateOperator(\n    \"CreateCounter\",\n    [],\n    [\"counter\"],\n    init_count=5\n)\n\nretrievecount_op = core.CreateOperator(\n    \"RetrieveCount\",\n    [\"counter\"],\n    [\"count\"]\n)\n\ncheckcounterdone_op = core.CreateOperator(\n    \"CheckCounterDone\",\n    [\"counter\"],\n    [\"done\"]\n)\n\ncountup_op = core.CreateOperator(\n    \"CountUp\",\n    [\"counter\"],\n    [\"previous_count\"],\n)\n\ncountdown_op = core.CreateOperator(\n    \"CountDown\",\n    [\"counter\"],\n    [\"done\"],\n)\n\nresetcounter_op = core.CreateOperator(\n    \"ResetCounter\",\n    [\"counter\"],\n    [\"previous_count\"],\n    init_count=3\n)\n\n\n// Create counter\nworkspace.RunOperatorOnce(createcounter_op)\nprint(\"'counter' pointer:\", workspace.FetchBlob(\"counter\"))\n\n\n// Retrieve initial counter value\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"Initial 'count':\", workspace.FetchBlob(\"count\"))\n\n\n// Check if counter is done\nworkspace.RunOperatorOnce(checkcounterdone_op)\nprint(\"Initial 'done' value:\", workspace.FetchBlob(\"done\"))\n\n\n// Test CountUp operator\nprint(\"\\nTesting CountUp operator...\")\nfor i in range(5):\n    workspace.RunOperatorOnce(countup_op)\n    print(\"'previous_count' after CountUp:\", workspace.FetchBlob(\"previous_count\"))\n\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"'count' value after CountUp test:\", workspace.FetchBlob(\"count\"))\n\n\n// Test CountDown operator\nprint(\"\\nTesting CountDown operator...\")\nfor i in range(11):\n    workspace.RunOperatorOnce(countdown_op)\n    workspace.RunOperatorOnce(retrievecount_op)\n    print(\"'count' value after CountDown: {}\\t'done' value: {}\".format(workspace.FetchBlob(\"count\"), workspace.FetchBlob(\"done\")))\n```\n\n**Result**\n\n```\n'counter' pointer: counter, a C++ native class of type std::__1::unique_ptr<caffe2::Counter<long long>, std::__1::default_delete<caffe2::Counter<long long> > >.\nInitial 'count': 5\nInitial 'done' value: False\n\nTesting CountUp operator...\n'previous_count' after CountUp: 5\n'previous_count' after CountUp: 6\n'previous_count' after CountUp: 7\n'previous_count' after CountUp: 8\n'previous_count' after CountUp: 9\n'count' value after CountUp test: 10\n\nTesting CountDown operator...\n'count' value after CountDown: 9        'done' value: False\n'count' value after CountDown: 8        'done' value: False\n'count' value after CountDown: 7        'done' value: False\n'count' value after CountDown: 6        'done' value: False\n'count' value after CountDown: 5        'done' value: False\n'count' value after CountDown: 4        'done' value: False\n'count' value after CountDown: 3        'done' value: False\n'count' value after CountDown: 2        'done' value: False\n'count' value after CountDown: 1        'done' value: False\n'count' value after CountDown: 0        'done' value: False\n'count' value after CountDown: -1        'done' value: True\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<ptr>`)* A blob pointing to an instance of a counter.",
          "name": "counter"
        }
      ],
      "outputs": [
        {
          "description": "*(type: int)* [OPTIONAL] count value BEFORE this operation.",
          "name": "previous_value"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "NormalizeGradient",
    "schema": {
      "attributes": [
        {
          "description": "axis to normalize",
          "name": "axis",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseMomentumSGDUpdate",
    "schema": {
      "attributes": [
        {
          "description": "Momentum hyperparameter.",
          "name": "momentum",
          "option": "optional"
        },
        {
          "description": "(boolean) Whether to use Nesterov Accelerated Gradient.",
          "name": "nesterov",
          "option": "optional"
        }
      ],
      "description": "\n\nPerforms a momentum SGD update analogous to MomentumSGDUpdate, but using a\nGradientSlice and indices into the full param and momentum tables. Both param\nand momentum should be in-place (corresponding inputs and outputs should be the\nsame blobs).\n\n\n\n",
      "inputs": [
        {
          "description": "GradientSlice with gradients for updated indices.",
          "name": "grad"
        },
        {
          "description": "Momentum blob, same shape as param.",
          "name": "moment"
        },
        {
          "description": "Learning rate.",
          "name": "lr"
        },
        {
          "description": "Full parameter blob.",
          "name": "param"
        },
        {
          "description": "Indices (in first dimension of param) where updates are performed.",
          "name": "indices"
        }
      ],
      "outputs": [
        {
          "description": "Adjusted gradient.",
          "name": "output_grad"
        },
        {
          "description": "Updated momentum.",
          "name": "output_moment"
        },
        {
          "description": "Updated parameter",
          "name": "output_param"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AffineChannelGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ColwiseMaxGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Conv2DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CosGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Scatter",
    "schema": {
      "attributes": [
        {
          "default": 1,
          "description": "Which dimension to scatter on.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nUpdate values of the tensor by overriding current value specified by indices.\n\nWrites all values from the tensor UPDATES into DATA at the indices specified in the INDICES tensor.\nFor each value in DATA, its output index is specified by its index in UPDATES and by the corresponding value in INDICES for the specified axis.\n\nFor a 3-D tensor, DATA is updated as:\n\nDATA[INDICES[i][j][k]][j][k] = UPDATES[i][j][k]  # if axis == 0\nDATA[i][INDICES[i][j][k]][k] = UPDATES[i][j][k]  # if axis == 1\nDATA[i][j][INDICES[i][j][k]] = UPDATES[i][j][k]  # if axis == 2\n\nCurrently only works on CPU because of access to INDICES.\n",
      "inputs": [
        {
          "description": "Tensor to be updated.",
          "name": "DATA"
        },
        {
          "description": "1-D list of indices on the first dimensionof X_0 that need to be updated",
          "name": "INDICES"
        },
        {
          "description": "Update slices, with shape len(INDICES) + shape(X_0)[1:]",
          "name": "UPDATES"
        }
      ],
      "outputs": [
        {
          "description": "The updated output.",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MapToKeyValue",
    "schema": {
      "description": "Convert a map blob into key and value blob pairs",
      "inputs": [
        {
          "description": "Blob reference to the map",
          "name": "map blob"
        }
      ],
      "outputs": [
        {
          "description": "Blob reference to the key",
          "name": "key blob"
        },
        {
          "description": "Blob reference to the value",
          "name": "value blob"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StringStartsWith",
    "schema": {
      "attributes": [
        {
          "description": "The prefix to check input strings against.",
          "name": "prefix",
          "option": "optional"
        }
      ],
      "description": "\nPerforms the starts-with check on each string in the input tensor.\nReturns tensor of boolean of the same dimension of input.\n",
      "inputs": [
        {
          "description": "Tensor of std::string.",
          "name": "strings"
        }
      ],
      "outputs": [
        {
          "description": "Tensor of bools of same shape as input.",
          "name": "bools"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "IndexStore",
    "schema": {
      "description": "\nStores the keys of this index in a 1-D tensor. Since element 0 is reserved\nfor unknowns, the first element of the output tensor will be element of index 1.\n",
      "inputs": [
        {
          "description": "Pointer to an Index instance.",
          "name": "handle"
        }
      ],
      "outputs": [
        {
          "description": "1-D tensor with elements starting with index 1.",
          "name": "items"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Im2Col",
    "schema": {
      "description": "The Im2Col operator from Matlab.",
      "inputs": [
        {
          "description": "4-tensor in NCHW or NHWC.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "4-tensor. For NCHW: N x (C x kH x kW) x outH x outW.For NHWC: N x outH x outW x (kH x kW x C",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FCTransposedGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "AddPadding",
    "schema": {
      "attributes": [
        {
          "description": "Number of copies of padding to add around each range.",
          "name": "padding_width",
          "option": "optional",
          "type": "int64"
        },
        {
          "description": "[OPTIONAL] Specifies a different end-padding width. If this is not set, will use same as `padding_width`.",
          "name": "end_padding_width",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nGiven a partitioned tensor $T<N, D_1, ..., D_n>$, where the partitions are\ndefined as ranges on its outer-most (slowest varying) dimension $N$,\nreturn a tensor $T<(N + 2 * padding\\_width), D_1, ..., D_n>$ with paddings\nadded to the start and end of each range.\n\nOptionally, different paddings can be provided for beginning and end.\nPaddings provided must be a tensor $T<D_1, ..., D_n>$. If no padding is\nprovided, add zero padding. If no lengths vector is provided, add padding\nonly once, at the start and end of data.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/sequence_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"AddPadding\",\n    [\"X\", \"lengths\"],\n    [\"Y\", \"lengths_out\"],\n    padding_width=1\n\n)\n\nworkspace.FeedBlob(\"X\", (np.random.rand(3,2,2).astype(np.float32)))\nworkspace.FeedBlob(\"lengths\", np.array([3]).astype(np.int32))\n\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\nprint(\"lengths_out:\", workspace.FetchBlob(\"lengths_out\"))\n```\n\n**Result**\n\n```\nX: [[[0.2531572  0.4588472 ]\n  [0.45140603 0.61161053]]\n\n [[0.92500854 0.8045306 ]\n  [0.03356671 0.30233648]]\n\n [[0.4660227  0.6287745 ]\n  [0.79372746 0.08609265]]]\nY: [[[0.         0.        ]\n  [0.         0.        ]]\n\n [[0.2531572  0.4588472 ]\n  [0.45140603 0.61161053]]\n\n [[0.92500854 0.8045306 ]\n  [0.03356671 0.30233648]]\n\n [[0.4660227  0.6287745 ]\n  [0.79372746 0.08609265]]\n\n [[0.         0.        ]\n  [0.         0.        ]]]\nlengths_out: [5]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor)* Input data ($T<N, D_1, ..., D_n>$).",
          "name": "data_in"
        },
        {
          "description": "*(type: Tensor`<int>`)* Number of elements in each range. sum(lengths) = N.",
          "name": "lengths"
        },
        {
          "description": "*(type: Tensor`<int>`)* [OPTIONAL] Padding data for range start ($T<D_1, ..., D_n>$).",
          "name": "start_padding"
        },
        {
          "description": "*(type: Tensor`<int>`)* [OPTIONAL] Padding for range end. If not provided, `start_padding` is used ($T<D_1, ..., D_n>$).",
          "name": "end_padding"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor)* Padded data tensor ($T<N + 2*padding_width, D_1, ..., D_n>$).",
          "name": "data_out"
        },
        {
          "description": "*(type: Tensor`<int>`)* [OPTIONAL] Lengths for each padded range.",
          "name": "lengths_out"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8Reshape",
    "schema": {
      "attributes": [
        {
          "description": "New shape",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "\nReshape the input tensor similar to numpy.reshape.\n\nIt takes a tensor as input and an optional tensor specifying the new shape.\nWhen the second input is absent, an extra argument `shape` must be specified.\nIt outputs the reshaped tensor as well as the original shape.\n\nAt most one dimension of the new shape can be -1. In this case, the value is\ninferred from the size of the tensor and the remaining dimensions. A dimension\ncould also be 0, in which case the actual dimension value is going to be copied\nfrom the input tensor.\n",
      "inputs": [
        {
          "description": "An input tensor.",
          "name": "data"
        },
        {
          "description": "New shape.",
          "name": "new_shape"
        }
      ],
      "outputs": [
        {
          "description": "Reshaped data.",
          "name": "reshaped"
        },
        {
          "description": "Original shape.",
          "name": "old_shape"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Where",
    "schema": {
      "description": "\nOperator Where takes three input data (Tensor, Tensor, Tensor) and\nproduces one output data (Tensor) where z = c ? x : y is applied elementwise.\n",
      "inputs": [
        {
          "description": "input tensor containing booleans",
          "name": "C"
        },
        {
          "description": "input tensor",
          "name": "X"
        },
        {
          "description": "input tensor",
          "name": "Y"
        }
      ],
      "outputs": [
        {
          "description": "output tensor",
          "name": "Z"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GetAllBlobNames",
    "schema": {
      "attributes": [
        {
          "description": "(bool, default true) Whether to include blobs inherited from parent workspaces.",
          "name": "include_shared",
          "option": "optional"
        }
      ],
      "description": "\nReturn a 1D tensor of strings containing the names\nof each blob in the active workspace.\n",
      "outputs": [
        {
          "description": "1D tensor of strings containing blob names.",
          "name": "blob_names"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FloatToFusedRandRowwiseQuantized",
    "schema": {
      "attributes": [
        {
          "description": "How many bits to quantize per data (defaults to 8).",
          "name": "bitwidth",
          "option": "optional"
        },
        {
          "description": "random or not (True). False is set up for unittest.",
          "name": "random",
          "option": "optional"
        }
      ],
      "description": "\nApplies row-wise stochastic/random quantization by determining the range of\neach row in the input matrix, and then quantize each element to one of two\nclosest discrete levels by randomly drawing Bernoulli distribution.\nThe method is extended from TernGrad [1],\nwhich randomly quantizes gradients to three levels to reduce communication in distributed training.\nThe format of each row (x) in the output matrix is [bitwidth][tail][min][max][data]:\nbitwidth[1 Byte]: bitwidth per data [1, 2, 4 or 8];\ntail[1 Byte]: the number of unused buckets [1-8] (One byte is split to 8/bitwidth buckets and each bucket stores one low-precision data in bitwidth bits);\nmin[4 Bytes]: the minimum floating value min(x);\nmax[4 Bytes]: the maximum floating value max(x);\ndata: quantized data.\nThe quantization is uniform with levels q = min + (max-min)/(2^bitwidth - 1)*[0:1:2^bitwidth].\nDuring stochastic/random quantization x'=Quantize(x), for q_j < x_i <= q_{j+1}, we draw quantization x'_i from Bernoulli distributions with\nP(x'_i = q_{j+1}) = (x_i - q_j)/(q_{j+1} - q_j), and\nP(x'_i = q_j) = (q_{j+1} - x_i)/(q_{j+1} - q_j) where x'_i is the quantized value of x_i.\n[1] proved E{x'_i}=x_i, which is an unbiased approximation. More details are in the paper.\nFor example, suppose targeted bitwidth = 2 and x = [0.3, -1.4, -0.6, 0.9, 1.0],\nthen tail = 3, min = -1.4, max = 1.0 and q = [-1.4, -0.6, 0.2, 1.0].\nx_1 = 0.3 will be quantized to x'_1 = 0.2 with probability 7/8 and to x'_1 = 1.0 with probability 1/8.\nThe storage format of quantized data is: [x'_1|x'_3|x'_5|xxx]-[x'_2|x'_4|xxx|xxx].\nIn general, a input row is split to multiple segments. One segment is a continuous subarray of the row,\nand its length is the number of bytes storing quantized data in the output matrix.\nThe b-th bucket of the i-th byte stores the i-th data of the b-th segment of input row.\n\n[1] Wen, Wei, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li.\n\"Terngrad: Ternary gradients to reduce communication in distributed deep learning.\"\nIn Advances in Neural Information Processing Systems, pp. 1508-1518. 2017.\n\n",
      "inputs": [
        {
          "description": "Float32 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused bitwidth, tail, min, max and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PackRNNSequence",
    "schema": {
      "description": "\nPack values based on the length blob. Each number from length blob represents\nthe corresponding values that need to be packed. The dimension for each pack\nis the same as the maximum number from the length blob (padding with zero is\nimplemented for smaller length value). The overall output dimension is:\nT * N * D, where T is the max number of lengths, N is the size of lengths,\nand D is the dimension of each feature value. The following example shows\nthe input and output of this operator:\n\n\nGiven:\n  values = [v1, v2, v3, v4, v5, v6, v7, v8]\n  lengths = [2, 3, 1, 2];\n\n\nOutput:\n  output = [\n    [v1, v3, v6, v7],\n    [v2, v4, 0,  v8],\n    [0,  v5, 0,  0 ],\n  ]\n\n\nOne application for this operator is the transfer data into the format that is\nused for RNN models. Note that the gradient operator of PackRNNSequence is\nUnpackRNNSequence.\n",
      "inputs": [
        {
          "description": "Data tensor, contains a sequence of features",
          "name": "values"
        },
        {
          "description": "lengths with each number representing the pack size.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor after packing",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PadEmptySamples",
    "schema": {
      "description": "\nPad empty field given lengths and index features,\n\nInput(0) is a blob pointing to the lengths of samples in one batch,\n[Input(1),... Input(num_fields)] a list of tensors containing the data for\neach field of the features.\n\nPadEmptySamples is thread safe.\n",
      "inputs": [
        {
          "description": "A blob containing a pointer to the lengths.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Tensor containing lengths with empty sample padded.",
          "name": "out_lengths"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PadImage",
    "schema": {
      "description": "\nPadImage pads values around the boundary of an image according to the pad\nvalues and stride sizes defined by the ConvPoolOpBase operator.\n  ",
      "inputs": [
        {
          "description": "Input data tensor from the previous operator; dimensions depend on whether the NCHW or NHWC operators are being used. For example, in the former, the input has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. The corresponding permutation of dimensions is used in the latter case. ",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output data tensor from padding the H and W dimensions on the tensor. Dimensions will vary based on various pad and stride sizes.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Glu",
    "schema": {
      "description": "\nApplies gated linear unit to the input Tensor X. The output Y is half the size\nof the input X, so if the shape of X is [d1, d2, ..., N] shape of Y will be\n[d1, d2, ..., dn/2] and Y(:dn-1, i) = GLU(X(:dn-1, i), X(:dn-1, i+N/2)) =\nX(dn-1, i) * sigmoid(X(dn-1, i+N/2))\n",
      "inputs": [
        {
          "description": "1D input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "1D output tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Shape",
    "schema": {
      "attributes": [
        {
          "description": "Array of interested axes.If given, this operator only returns the dimensions of the given axes.Otherwise, the operator returns the dimensions of all axes.",
          "name": "axes",
          "option": "optional",
          "type": "int64[]"
        }
      ],
      "description": "\nProduce a 1D int64 tensor with the shape of the input tensor.\nIf called with an optional argument `axes`, the result will only\ncontain the dimensions of specified axes.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/shape_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Shape\",\n    [\"X\"],\n    [\"shape\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.randint(10, size=(2,3))))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"shape:\", workspace.FetchBlob(\"shape\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[3 2 5]\n [5 7 3]]\nshape: [2 3]\n\n```\n\n</details>\n\n      ",
      "inputs": [
        {
          "description": "*(type: Tensor)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor)* Output tensor containing shape of input tensor.",
          "name": "shape"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReservoirSampling",
    "schema": {
      "attributes": [
        {
          "description": "The number of random samples to append for each positive samples",
          "name": "num_to_collect",
          "option": "optional"
        }
      ],
      "description": "\nCollect `DATA` tensor into `RESERVOIR` of size `num_to_collect`. `DATA` is\nassumed to be a batch.\n\nIn case where 'objects' may be repeated in data and you only want at most one\ninstance of each 'object' in the reservoir, `OBJECT_ID` can be given for\ndeduplication. If `OBJECT_ID` is given, then you also need to supply additional\nbook-keeping tensors. See input blob documentation for details.\n\nThis operator is thread-safe.\n",
      "inputs": [
        {
          "description": "The reservoir; should be initialized to empty tensor",
          "name": "RESERVOIR"
        },
        {
          "description": "Number of examples seen so far; should be initialized to 0",
          "name": "NUM_VISITED"
        },
        {
          "description": "Tensor to collect from. The first dimension is assumed to be batch size. If the object to be collected is represented by multiple tensors, use `PackRecords` to pack them into single tensor.",
          "name": "DATA"
        },
        {
          "description": "Mutex to prevent data race",
          "name": "MUTEX"
        },
        {
          "description": "(Optional, int64) If provided, used for deduplicating object in the reservoir",
          "name": "OBJECT_ID"
        },
        {
          "description": "(Optional) Auxiliary bookkeeping map. This should be created from  `CreateMap` with keys of type int64 and values of type int32",
          "name": "OBJECT_TO_POS_MAP_IN"
        },
        {
          "description": "(Optional) Tensor of type int64 used for bookkeeping in deduplication",
          "name": "POS_TO_OBJECT_IN"
        }
      ],
      "outputs": [
        {
          "description": "Same as the input",
          "name": "RESERVOIR"
        },
        {
          "description": "Same as the input",
          "name": "NUM_VISITED"
        },
        {
          "description": "(Optional) Same as the input",
          "name": "OBJECT_TO_POS_MAP"
        },
        {
          "description": "(Optional) Same as the input",
          "name": "POS_TO_OBJECT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BatchBoxCox",
    "schema": {
      "description": "\nInput `data` is a N * D matrix. Apply box-cox transform for each column.\n`lambda1` and `lambda2` is of size D that defines the hyper-parameters for\nthe transform of each column `x` of the input `data`:\n\n    ln(x + lambda2), if lambda1 == 0\n    ((x + lambda2)^lambda1 - 1)/lambda1, if lambda1 != 0\n\n",
      "inputs": [
        {
          "description": "input float or double N * D matrix",
          "name": "data"
        },
        {
          "description": "tensor of size D with the same type as data",
          "name": "lambda1"
        },
        {
          "description": "tensor of size D with the same type as data",
          "name": "lambda2"
        }
      ],
      "outputs": [
        {
          "description": "output matrix that applied box-cox transform",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Clip",
    "schema": {
      "attributes": [
        {
          "description": "Minimum value, under which element is replaced by min (default=*numeric_limits::lowest()*).",
          "name": "min",
          "option": "optional",
          "type": "float32"
        },
        {
          "description": "Maximum value, under which element is replaced by max (default=*numeric_limits::max()*).",
          "name": "max",
          "option": "optional",
          "type": "float32"
        }
      ],
      "description": "\nThis operator limits the given input within an interval. The interval is\nspecified by the `min` and `max` arguments. They default to\n*numeric_limits::lowest()* and *numeric_limits::max()* respectively. The\nclipping operation can be done in an in-place fashion by using the same output\nblob as the input blob.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/clip_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Clip\",\n    [\"X\"],\n    [\"Y\"],\n    min=20.0,\n    max=60.0\n\n)\n\nworkspace.FeedBlob(\"X\", (np.random.randint(100, size=(5,5))).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\nX: [[45. 16. 59. 99. 48.]\n [12. 44. 46. 82. 28.]\n [ 1. 91. 18.  9. 71.]\n [24. 37. 61. 12. 81.]\n [36. 38. 30. 84. 40.]]\nY: [[45. 20. 59. 60. 48.]\n [20. 44. 46. 60. 28.]\n [20. 60. 20. 20. 60.]\n [24. 37. 60. 20. 60.]\n [36. 38. 30. 60. 40.]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(Tensor`<float>`)* Input tensor within range [*numeric_limits::lowest()*, *numeric_limits::max()*].",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(Tensor`<float>`)* Output tensor clipped within range [`min`, `max`].",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FeedBlob",
    "schema": {
      "attributes": [
        {
          "description": "(string) if provided then we will use this string as the value for theprovided output tensor",
          "name": "value",
          "option": "optional"
        }
      ],
      "description": "\nFeedBlobs the content of the blobs. The input and output blobs should be\none-to-one inplace.",
      "support_level": "default"
    }
  },
  {
    "name": "CreateBlobsQueue",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "BisectPercentile",
    "schema": {
      "attributes": [
        {
          "description": "1D tensor, which is the concatenation of all sorted raw feature values for all features.",
          "name": "percentile_raw",
          "option": "optional"
        },
        {
          "description": "1D tensor. There is one-one mapping between percentile_mapping and percentile_raw such that each element in percentile_mapping corresponds to the percentile value of the corresponding raw feature value.",
          "name": "percentile_mapping",
          "option": "optional"
        },
        {
          "description": "1D tensor. There is one-one mapping between percentile_upper and percentile_raw such that each element in percentile_mapping corresponds to the percentile lower bound of the corresponding raw feature value.",
          "name": "percentile_lower",
          "option": "optional"
        },
        {
          "description": "1D tensor. There is one-one mapping between percentile_upper and percentile_raw such that each element in percentile_mapping corresponds to the percentile upper bound of the corresponding raw feature value.",
          "name": "percentile_upper",
          "option": "optional"
        },
        {
          "description": "1D tensor. There is one-one mapping between percentile_upper and percentile_raw such that each element in percentile_mapping corresponds to the percentile upper bound of the corresponding raw feature value.",
          "name": "lengths",
          "option": "optional"
        }
      ],
      "description": "\n    This operator is to map raw feature values into the percentile\n    representations based on Bisection for more than one feature.\n\n    The input is the bath of input feature values, with the size of (batch_size,\n    num_feature), where num_feature = F (F >= 1).\n\n    For each feature, we also need additional information regarding the feature\n    value distribution.\n    There are several vectors to keep data to percentile mappping information\n    as arguments (context):\n    1. feature raw values (R)\n    2. feature percentile mapping (P)\n    3. feature percentile lower bound (L)\n    4. feature percentile upper bound (U)\n\n    A toy example:\n    Suppose the sampled data distribution is as follows:\n    1, 1, 2, 2, 2, 2, 2, 2, 3, 4\n    We have the mapping vectors as follows:\n    R = [1, 2, 3, 4]\n    P = [0.15, 0.55, 0.9, 1.0]\n    L = [0.1, 0.3, 0.9, 1.0]\n    U = [0.2, 0.8, 0.9, 1.0]\n    Where P is computed as (L + U) / 2.\n\n    For a given list of feature values, X = [x_0, x_1, ..., x_i, ...], for each\n    feature value (x_i) we first apply bisection to find the right index (t),\n    such that R[t] <= x_i < R[t+1].\n    If x_i = R[t], P[t] is returned;\n    otherwise, the interpolation is apply by (R[t], R[t+1]) and (U[t] and L[t]).\n\n    As there are F features (F >= 1), we concate all the R_f, P_f, L_f, and\n    U_f for each feature f and use an additional input length to keep track of\n    the number of points for each set of raw feature value to percentile mapping.\n    For example, there are two features:\n    R_1 =[0.1, 0.4, 0.5];\n    R_2 = [0.3, 1.2];\n    We will build R = [0.1, 0.4, 0.5, 0.3, 1.2]; besides, we have\n    lengths = [3, 2]\n    to indicate the boundaries of the percentile information.\n\n",
      "inputs": [
        {
          "description": "Input 2D tensor of floats of size (N, D), where N is the batch size and D is the feature dimension.",
          "name": "raw_values"
        }
      ],
      "outputs": [
        {
          "description": "2D tensor of output with the same dimensions as the input raw_values.",
          "name": "percentile"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReversePackedSegs",
    "schema": {
      "description": "\nReverse segments in a 3-D tensor (lengths, segments, embeddings,), leaving\npaddings unchanged. This operator is used to reverse input of a recurrent neural\nnetwork to make it a BRNN.\n  ",
      "inputs": [
        {
          "description": "a 3-D (lengths, segments, embeddings,) tensor.",
          "name": "data"
        },
        {
          "description": "length of each segment.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "a (lengths, segments, embeddings,) tensor with each segment reversedand paddings unchanged.",
          "name": "reversed data"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CreateScope",
    "schema": {
      "description": "\n'CreateScope' operator initializes and outputs empty scope that is used\nby Do operator to store local blobs\n    ",
      "support_level": "default"
    }
  },
  {
    "name": "SpatialSoftmaxWithLossGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "StoreAdd",
    "schema": {
      "attributes": [
        {
          "description": "key of the counter (required)",
          "name": "blob_name",
          "option": "optional"
        },
        {
          "description": "value that is added (optional, default: 1)",
          "name": "add_value",
          "option": "optional"
        }
      ],
      "description": "\nAdd a value to a remote counter. If the key is not set, the store\ninitializes it to 0 and then performs the add operation. The operation\nreturns the resulting counter value.\n",
      "inputs": [
        {
          "description": "unique_ptr<StoreHandler>",
          "name": "handler"
        }
      ],
      "outputs": [
        {
          "description": "the current value of the counter",
          "name": "value"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MergeSingleListFeatureTensorsGradient",
    "schema": {
      "description": "Explode multi-feature tensors with list features into single-feature tensors.\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": ".lengths",
          "name": "in1_lengths"
        },
        {
          "description": ".presence",
          "name": "in1_presence"
        },
        {
          "description": ".values.values_grad",
          "name": "out_values_values"
        }
      ],
      "outputs": [
        {
          "description": ".values_grad",
          "name": "out1_values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "DistributeFpnProposals",
    "schema": {
      "attributes": [
        {
          "description": "(int) ROI_CANONICAL_SCALE",
          "name": "roi_canonical_scale",
          "option": "optional"
        },
        {
          "description": "(int) ROI_CANONICAL_LEVEL",
          "name": "roi_canonical_level",
          "option": "optional"
        },
        {
          "description": "(int) ROI_MAX_LEVEL",
          "name": "roi_max_level",
          "option": "optional"
        },
        {
          "description": "(int) ROI_MIN_LEVEL",
          "name": "roi_min_level",
          "option": "optional"
        }
      ],
      "description": "\n...\n",
      "inputs": [
        {
          "description": "Top proposals limited to rpn_post_nms_topN total, format (image_index, x1, y1, x2, y2)",
          "name": "rois"
        }
      ],
      "outputs": [
        {
          "description": "RPN proposals for ROI level 2, format (image_index, x1, y1, x2, y2)",
          "name": "rois_fpn2"
        },
        {
          "description": "RPN proposals for ROI level 3, format (image_index, x1, y1, x2, y2)",
          "name": "rois_fpn3"
        },
        {
          "description": "RPN proposals for ROI level 4, format (image_index, x1, y1, x2, y2)",
          "name": "rois_fpn4"
        },
        {
          "description": "RPN proposals for ROI level 5, format (image_index, x1, y1, x2, y2)",
          "name": "rois_fpn5"
        },
        {
          "description": "Permutation on the concatenation of all rois_fpni, i=min...max, such that when applied the RPN RoIs are restored to their original order in the input blobs.",
          "name": "rois_idx_restore"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CollectRpnProposals",
    "schema": {
      "attributes": [
        {
          "description": "(int) RPN_MAX_LEVEL",
          "name": "rpn_max_level",
          "option": "optional"
        },
        {
          "description": "(int) RPN_MIN_LEVEL",
          "name": "rpn_min_level",
          "option": "optional"
        },
        {
          "description": "(int) RPN_POST_NMS_TOP_N",
          "name": "rpn_post_nms_topN",
          "option": "optional"
        }
      ],
      "description": "\n...\n",
      "inputs": [
        {
          "description": "RPN proposals for FPN level 2, format (image_index, x1, y1, x2, y2). See rpn_rois documentation from GenerateProposals.",
          "name": "rpn_rois_fpn2"
        },
        {
          "description": "RPN proposals for FPN level 3, format (image_index, x1, y1, x2, y2). See rpn_rois documentation from GenerateProposals.",
          "name": "rpn_rois_fpn3"
        },
        {
          "description": "RPN proposals for FPN level 4, format (image_index, x1, y1, x2, y2). See rpn_rois documentation from GenerateProposals.",
          "name": "rpn_rois_fpn4"
        },
        {
          "description": "RPN proposals for FPN level 5, format (image_index, x1, y1, x2, y2). See rpn_rois documentation from GenerateProposals.",
          "name": "rpn_rois_fpn5"
        },
        {
          "description": "RPN proposals for FPN level 6, format (image_index, x1, y1, x2, y2). See rpn_rois documentation from GenerateProposals.",
          "name": "rpn_rois_fpn6"
        },
        {
          "description": "RPN objectness probabilities for FPN level 2. See rpn_roi_probs documentation from GenerateProposals.",
          "name": "rpn_roi_probs_fpn2"
        },
        {
          "description": "RPN objectness probabilities for FPN level 3. See rpn_roi_probs documentation from GenerateProposals.",
          "name": "rpn_roi_probs_fpn3"
        },
        {
          "description": "RPN objectness probabilities for FPN level 4. See rpn_roi_probs documentation from GenerateProposals.",
          "name": "rpn_roi_probs_fpn4"
        },
        {
          "description": "RPN objectness probabilities for FPN level 5. See rpn_roi_probs documentation from GenerateProposals.",
          "name": "rpn_roi_probs_fpn5"
        },
        {
          "description": "RPN objectness probabilities for FPN level 6. See rpn_roi_probs documentation from GenerateProposals.",
          "name": "rpn_roi_probs_fpn6"
        }
      ],
      "outputs": [
        {
          "description": "Top proposals limited to rpn_post_nms_topN total, format (image_index, x1, y1, x2, y2)",
          "name": "rois"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsMaxWithMainInputAndForwardOutputGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "MomentumSGDUpdate",
    "schema": {
      "description": "\n\nPerforms a momentum SGD update for an input gradient and momentum\nparameters. Concretely, given inputs (grad, m, lr, param) and arguments\n(momentum, nesterov), computes:\n\n    if not nesterov:\n        adjusted_gradient = lr * grad + momentum * m\n        param = param - adjusted_gradient\n        return (adjusted_gradient, adjusted_gradient, param)\n    else:\n        m_new = momentum * m + lr * grad\n        param = param - ((1 + momentum) * m_new - momentum * m),\n        return ((1 + momentum) * m_new - momentum * m, m_new, param)\n\nOutput is (grad, momentum, parameter).\n\nNote the difference to MomentumSGD, which returns a new gradient\nbut does not perform the parameter update.\n\n",
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsIndicesInGradientWeightedSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Pow",
    "schema": {
      "attributes": [
        {
          "description": "The exponent of the power function. Do not use if setting exponent via input.",
          "name": "exponent",
          "option": "optional"
        },
        {
          "default": -1,
          "description": "",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": false,
          "description": "",
          "name": "broadcast",
          "option": "optional",
          "type": "boolean"
        }
      ],
      "description": "\nThe *Pow* op takes an input data tensor $X$ and an exponent parameter *exponent*, which can be a scalar or another tensor. As output, it produces a single output data tensor $Y$, where the function $f(x) = x^{exponent}$ has been applied to $X$ elementwise.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pow_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pow_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Pow\",\n    [\"X\", \"exponent\"],\n    [\"Y\"],\n    broadcast=1\n)\n\nworkspace.FeedBlob(\"X\", np.array([1,2,3,4,5,6]).astype(np.float32))\nprint(\"X: \", workspace.FetchBlob(\"X\"))\n\nworkspace.FeedBlob(\"exponent\", np.array([2]).astype(np.float32))\nprint(\"exponent: \", workspace.FetchBlob(\"exponent\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"Y: \", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:  [1. 2. 3. 4. 5. 6.]\nexponent:  [2.]\nY:  [ 1.  4.  9. 16. 25. 36.]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "Input data blob to be operated on.",
          "name": "X"
        },
        {
          "description": "Exponent blob containing the exponent(s) for calculation. Do not use if setting exponent via argument.",
          "name": "exponent"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob with the same shape as the input.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Cube",
    "schema": {
      "description": null,
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor calculated as the cube of the input tensor, element-wise.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AveragePool2D",
    "schema": {
      "description": "AveragePool2D \nconsumes an input blob and applies average pooling across the the blob according\nto kernel sizes, stride sizes, pad lengths and dilation. Average pooling consists\nof taking the average value of a subset of the input tensor according to the kernel\nsize and downsampling the data into the output blob for further processing. The\n`brew` module has a wrapper for this operator for use in a `ModelHelper` object.\n\nPooling layers reduce the spatial dimensionality of the input blob. Each of the\noutput blob's dimensions will reduce according to:\n\n$$dim_{out}=\\frac{dim_{in}-kernel+2*pad}{stride}+1$$\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"AveragePool\",\n    [\"X\"],\n    [\"Y\"],\n    kernel=2,\n    stride=2,\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(1, 1, 6, 6).astype(np.float32)) // NCHW\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX:\n [[[[-0.2883434   0.43498734  0.05417408  1.912558    0.09390241\n    -0.33173105]\n   [ 1.633709    1.2047161   0.36964908  0.99961185  0.4184147\n     0.9989975 ]\n   [ 1.7644193   0.1789665   1.5812988  -0.6038542  -0.36090398\n     0.33195344]\n   [ 0.9457722  -0.95174325 -0.78124577  1.2062047   1.1903144\n     0.2586746 ]\n   [ 1.252104    0.32645547  1.8073524  -0.78397465  0.9978303\n    -0.97614396]\n   [ 0.5440196   1.5778259  -0.76750124  0.5051756   0.8838398\n    -0.37085298]]]]\n\nY:\n [[[[0.7462672  0.83399826 0.2948959 ]\n   [0.4843537  0.3506009  0.35500962]\n   [0.9251013  0.19026303 0.13366827]]]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor of shape NCHW or NHWC.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output data tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BitwiseAnd",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise bitwise operation `bitwise_and` (with limited broadcast support).\nBoth input operands should be of type `bool`.\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor)* First operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor)* Output tensor. Has same dimensions as input `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Mean",
    "schema": {
      "description": "\nElement-wise mean of an arbitrary number of input tensors. This operation can be\nperformed in-place, by using the first input blob as the output blob. All inputs\nmust have the same shape and data type, and the output will have the same shape\nas the inputs.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/mean_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Mean\",\n    [\"X\", \"Y\", \"Z\"],\n    [\"X\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.rand(3,3)).astype(np.float32))\nworkspace.FeedBlob(\"Y\", (np.random.rand(3,3)).astype(np.float32))\nworkspace.FeedBlob(\"Z\", (np.random.rand(3,3)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\nprint(\"Z:\", workspace.FetchBlob(\"Z\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Mean:\", workspace.FetchBlob(\"X\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[0.6035237  0.5305746  0.6298913 ]\n [0.9169737  0.01280353 0.16286302]\n [0.6017664  0.9946255  0.05128575]]\nY:\n[[0.07544111 0.45371833 0.08460239]\n [0.9708728  0.7422064  0.7933344 ]\n [0.97671497 0.3411384  0.73818344]]\nZ:\n[[0.08837954 0.90187573 0.46734726]\n [0.6308827  0.8719029  0.39888734]\n [0.90059936 0.92883426 0.5695987 ]]\nMean:\n[[0.25578147 0.6287229  0.39394698]\n [0.8395764  0.5423043  0.45169494]\n [0.8263602  0.75486606 0.45302266]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<Ord>`)* List of input tensors with the same shape.",
          "name": "X, Y, ..."
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<Ord>`)* Output tensor with the same dimensions as inputs. Contains the mean values of the input tensors calculated element-wise.",
          "name": "M"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "And",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise logical operation **and** (with limited broadcast support).\nBoth input operands should be of type `bool`.\n\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"And\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", (np.random.rand(3, 3) > 0.5))\nworkspace.FeedBlob(\"B\", (np.random.rand(3, 3) > 0.5))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\n\nA:\n [[ True False False]\n [False  True False]\n [False False  True]]\nB:\n [[ True False  True]\n [False False False]\n [False False False]]\nC:\n [[ True False False]\n [False False False]\n [False False False]]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "*(type: Tensor`<bool>`)* First operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<bool>`)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<bool>`)* Output tensor of booleans. Has same dimensions as input `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GivenTensorFill",
    "schema": {
      "attributes": [
        {
          "description": "The value of the elements to go in the *output* tensor.",
          "name": "values"
        },
        {
          "description": "The data type for the elements of the output tensor. Strictly must be one of the types from DataType enum in TensorProto.",
          "name": "dtype",
          "option": "optional"
        },
        {
          "description": "Desired shape of the *output* tensor.",
          "name": "shape",
          "option": "optional",
          "type": "int64[]"
        },
        {
          "description": "The additional dimensions appended at the end of the *shape* indicated by the input blob. Cannot set the *extra_shape* argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional",
          "type": "int64[]"
        },
        {
          "default": false,
          "description": "set to *True* to use the *input* as shape. First, input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional",
          "type": "boolean"
        }
      ],
      "description": "\nThis op fills an output tensor with the data specified by the *value* and *dtype* arguments.  The output tensor shape is specified by the *shape* argument. Beware, when using this argument *value* should have a value for every element of the *output*, as missing values will not be initialized automatically. If *input_as_shape* is set to *true*, then the *input* should be a 1D tensor containing the desired output shape (the dimensions specified in *extra_shape* will also be appended). In this case, the *shape* argument should **not** be set.\n\n*Note: Do not set the shape argument and pass in an input at the same time.*\n\nGithub Links:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/given_tensor_fill_op.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/given_tensor_fill_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"GivenTensorFill\",\n    [],\n    [\"out\"],\n    values=[1., 2., 3.],\n    shape=[3],\n)\n\nworkspace.RunOperatorOnce(op)\nprint(\"Out:\\n\", workspace.FetchBlob(\"out\"))\n\n```\n\n**Result**\n\n```\n\nOut:\n [1. 2. 3.]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(Optional) 1D tensor specifying the shape of the output. Must be used with *input_as_shape=True*",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor with desired dimension filled with specified data. If the shape argument is set, this is the shape specified, and if the *input* exists and *input_as_shape=True*, it is the shape specified by the *input* tensor.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GeluGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LayerNorm",
    "schema": {
      "attributes": [
        {
          "description": "(int) default to 1; Describes axis of the inputs. Defaults to one because the 0th axis most likely describes the batch size",
          "name": "axis",
          "option": "optional"
        },
        {
          "description": "(float) default to 0.001. Small value to be added to the stdev when dividing out by that value. This prevents division by zero.",
          "name": "epsilon",
          "option": "optional"
        },
        {
          "description": "(bool) default to False; If true, this op will do affine transformation after normalization.",
          "name": "elementwise_affine",
          "option": "optional"
        }
      ],
      "description": "\nComputes layer normalization as described in https://arxiv.org/pdf/1607.06450.pdf.\nGiven an input vector x \\in [a_0, a_1, ...,a_{k-1}, a_k, ..., a_{n-1}],\nthis op treats dimensions a_k through a_{n-1} as feature vectors. For each\nfeature vector, the op contains the mean and standard deviation. Then,\nit returns the normalized values (with respect to the feature vector).\n\nNote that this op does not contain the scale an bias terms described in the\npaper. Simply follow this op with an FC op to add those. Concretely, this op\nimplements:\n\nh = \\frac{1}{\\sigma}(a - \\mu)\nwhere \\mu = \\frac{1}{H}\\sum_{i=1}^{H} a_i\nand \\sigma = \\sqrt{\\frac{1}{H}\\sum_{i=1}^{H}(a_i - \\mu)^2}\nwhere H is the number of hidden units (i.e. product of dimensions from 'axis'\nto the end.)\n",
      "inputs": [
        {
          "description": "Input tensor which layer normalization will be applied to",
          "name": "input"
        },
        {
          "description": "scale tensor for elementwise_affine, the shape should be the same as the dimensions of X begin from axis",
          "name": "gamma"
        },
        {
          "description": "bias tensor for elementwise_affine, the shape should be the same as the dimensions of X begin from axis",
          "name": "beta"
        }
      ],
      "outputs": [
        {
          "description": "Normalized values",
          "name": "output"
        },
        {
          "description": "Mean values for each feature vector",
          "name": "mean"
        },
        {
          "description": "Standard deviations for each feature vector",
          "name": "stddev"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentWeightedSum",
    "schema": {
      "attributes": [
        {
          "description": "Produce also gradient for `weights`. For now it's only supported in `Lengths`-based operators",
          "name": "grad_on_weights",
          "option": "optional"
        }
      ],
      "description": "\nApplies 'WeightedSum' to each segment of input tensor. Segments need to be sorted and\ncontiguous. See also UnsortedSegmentWeightedSum that doesn't have this requirement.\n\nSEGMENT_IDS is a vector that maps each of the first dimension slices of the\nDATA to a particular group (segment). Values belonging to the same segment are\naggregated together.\n\nThe first dimension of the output is equal to the number of input segments,\ni.e. `SEGMENT_IDS[-1]+1`. Other dimensions are inherited from the input tensor.\n\nInput slices are first scaled by SCALARS and then summed element-wise. It doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor for the summation",
          "name": "DATA"
        },
        {
          "description": "Scalar multipliers for the input slices. Must be a vector with the length matching the number of slices",
          "name": "SCALARS"
        },
        {
          "description": "Vector with the same length as the first dimension of DATA and values in the range 0..K-1 and in increasing order that maps each slice of DATA to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of K (the number of segments).",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BernoulliJSD",
    "schema": {
      "description": "\nComputes the Jensen-Shannon divergence (JSD) between two Bernoulli distributions\nwhere each is parametrized by a single probability.\n",
      "inputs": [
        {
          "description": "array of probabilities for target",
          "name": "T"
        }
      ],
      "outputs": [
        {
          "description": "array of JSD losses",
          "name": "L"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Cosh",
    "schema": {
      "description": "\nCalculates the hyperbolic cosine of the given input tensor, element-wise.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/cosh_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Cosh\",\n    [\"X\"],\n    [\"Y\"]\n)\n\nworkspace.FeedBlob(\"X\", np.random.rand(5).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX: [0.66423494 0.32074615 0.81523746 0.90423071 0.39275789]\nY: [1.22883528 1.05188156 1.35112322 1.43744212 1.07812598]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The hyperbolic cosine values of the input tensor, computed element-wise",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceFrontWeightedSum",
    "schema": {
      "attributes": [
        {
          "description": "Produce also gradient for `weights`. For now it's only supported in `Lengths`-based operators",
          "name": "grad_on_weights",
          "option": "optional"
        }
      ],
      "description": "\nReduces the input tensor along the first dimension of the input tensor by\napplying 'WeightedSum'. This op acts in a similar way to SortedSegmentWeightedSum and\nUnsortedSegmentWeightedSum but as if all input slices belong to a single segment.\n\nInput slices are first scaled by SCALARS and then summed element-wise. It doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor for the summation",
          "name": "DATA"
        },
        {
          "description": "Scalar multipliers for the input slices. Must be a vector with the length matching the number of slices",
          "name": "SCALARS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated tensor",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "UnpackRNNSequence",
    "schema": {
      "description": "\nThis is the reverse operator for PackRNNSequence. It maps the packed values\nback to sequence values based on the length blob. Each number from length blob\nrepresents the corresponding values that has been grouped. The dimension\nfor each pack is the same as the maximum number from the length blob (padding\nwith zero was implemented for smaller length value). The overall output\ndimension is: M * D, where M is the sum of lengths, and D is the dimension of\neach feature value. The following example shows the input and output of\nthis operator:\n\n\nGiven:\n  values = [\n    [v1, v3, v6, v7],\n    [v2, v4, 0,  v8],\n    [0,  v5, 0,  0 ],\n  ]\n  lengths = [2, 3, 1, 2]\n\n\nOutput:\n  output = [v1, v2, v3, v4, v5, v6, v7, v8];\n\n\nOne application for this operator is the transfer data from the format of RNN\nback to sequence values. Note that the gradient operator of\nUnpackRNNSequence is PackRNNSequence.\n",
      "inputs": [
        {
          "description": "Data tensor, contains the packed features",
          "name": "values"
        },
        {
          "description": "lengths with each number representing the pack size.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor before packing",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ScaleBlobs",
    "schema": {
      "attributes": [
        {
          "description": "(float, default 1.0) the scale to apply.",
          "name": "scale",
          "option": "optional"
        }
      ],
      "description": "\nScaleBlobs takes one or more input data (Tensor) and produces one\nor more output data (Tensor) whose value is the input data tensor\nscaled element-wise.\n",
      "support_level": "default"
    }
  },
  {
    "name": "SafeEnqueueBlobs",
    "schema": {
      "description": "\nEnqueue the blobs into queue. When the queue is closed and full, the output\nstatus will be set to true which can be used as exit criteria for execution\nstep.\nThe 1st input is the queue and the last output is the status. The rest are\ndata blobs.\n",
      "inputs": [
        {
          "description": "The shared pointer for the BlobsQueue",
          "name": "queue"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "UnsortedSegmentSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Alias",
    "schema": {
      "description": "\nMakes the output and the input share the same underlying storage.\n\nWARNING: in general, in caffe2's operator interface different tensors should\nhave different underlying storage, which is the assumption made by\ncomponents such as the dependency engine and memory optimization. Thus, in\nnormal situations you should not use the AliasOp, especially in a normal\nforward-backward pass.\n\nThe Alias op is provided so one can achieve true asynchrony, such as\nHogwild, in a graph. But make sure you understand all the implications\nsimilar to multi-thread computation before you use it explicitly.\n",
      "inputs": [
        {
          "description": "Input tensor whose storage will be shared.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Tensor of same shape as input, sharing its storage.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ScatterWeightedSum",
    "schema": {
      "description": "\nSimilar to WeightedSum, computes the weighted sum of several tensors, with\nthe difference that inputs are sliced tensors. The first tensor has to be\nin-place and only slices of it on the first dimension as indexed by INDICES\nwill be updated.\n\nNote: The op pretty much ignores the exact shapes of the input arguments and\ncares only about sizes. It's done for performance consideration to avoid\nunnecessary reshapes. Only first dimension of X_0 is important, let's call it\nN. If M is the total size of X_0 and K is the size of INDICES then X_i is\nassumed to be of shape K x (M / N) regardless of the real shape.\n\nNote: Each update in INDICES is applied independently which means that if\nduplicated elements are present in INDICES the corresponding slice of X_0\nwill be scaled multiple times. Manual collapsing of INDICES is required\nbeforehand if necessary.\n\nNote: Updates are applied sequentially by inputs which might have undesired\nconsequences if the input tensor is accessed concurrently by different op\n(e.g. when doing Hogwild). Other threads might see intermediate results even\non individual slice level, e.g. X_0 scaled by weight_0 but without any\nupdates applied.\n\nCurrently only works on CPU because of access to INDICES.\n",
      "inputs": [
        {
          "description": "Tensor to be updated.",
          "name": "X_0"
        },
        {
          "description": "Scalar weight for X_0, applied only to slices affected.",
          "name": "Weight_0"
        },
        {
          "description": "1-D list of indices on the first dimension of X_0 that need to be updated",
          "name": "INDICES"
        },
        {
          "description": "Update slices, with shape len(INDICES) + shape(X_0)[1:]",
          "name": "X_1"
        },
        {
          "description": "Scalar weight for X_1 update",
          "name": "Weight_1"
        }
      ],
      "outputs": [
        {
          "description": "Has to be exactly the same tensor as the input 0",
          "name": "X_0"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8Flatten",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        },
        {
          "description": "(Default to 1) Indicate up to which input dimensions (exclusive) should be flattened to the outer dimension of the output",
          "name": "axis",
          "option": "optional"
        }
      ],
      "description": "\nFlattens the input tensor into a 2D matrix. If input tensor has shape\n(d_0, d_1, ... d_n) then the output will have shape\n(d_0 X d_1 ... d_(axis-1), d_axis X d_(axis+1) ... X dn)\n",
      "inputs": [
        {
          "description": "A Int8 tensor of rank >= axis.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "A 2D Int8 tensor with the contents of the input tensor, with input dimensions up to axis flattened to the outer dimension of the output and remaining input dimensions flattened into the inner dimension of the output.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LRNGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "MaxPool2D",
    "schema": {
      "description": "MaxPool2D \nconsumes an input blob and applies max pooling across the the blob according to\nkernel sizes, stride sizes, pad lengths and dilation. Max pooling consists of\ntaking the maximum value of a subset of the input tensor according to the kernel\nsize and downsampling the data into the output blob for further processing. The\n`brew` module has a wrapper for this operator for use in a `ModelHelper` object.\n\nPooling layers reduce the spatial dimensionality of the input blob. Each of the\noutput blob's dimensions will reduce according to:\n\n$$dim_{out}=\\frac{dim_{in}-kernel+2*pad}{stride}+1$$\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"MaxPool\",\n    [\"X\"],\n    [\"Y\"],\n    kernel=2,\n    stride=2,\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(1, 1, 6, 6).astype(np.float32)) // NCHW\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX:\n [[[[-2.8534958e-01 -1.7719941e+00 -8.2277227e-04  1.1088650e+00\n    -2.1476576e+00 -3.5070452e-01]\n   [-9.0058845e-01 -3.0070004e-01 -1.7907504e+00 -7.1746534e-01\n     1.2798511e+00 -3.2214901e-01]\n   [ 1.5806322e+00  1.6845188e+00 -2.6633200e-01 -3.8576153e-01\n    -9.6424848e-02 -3.9696163e-01]\n   [ 1.2572408e-01  6.3612902e-01 -3.9554062e-01 -6.9735396e-01\n    -9.1898698e-01 -1.9609968e-01]\n   [-1.1587460e+00  2.4605224e+00 -1.5497679e+00  1.3020347e-01\n    -8.1293899e-01 -7.8803545e-01]\n   [ 1.4323474e+00  1.3618395e+00  9.8975077e-02 -1.1307785e-01\n     7.2035044e-01  2.7642491e-01]]]]\n\nY:\n [[[[-0.28534958  1.108865    1.2798511 ]\n   [ 1.6845188  -0.266332   -0.09642485]\n   [ 2.4605224   0.13020347  0.72035044]]]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor of shape NCHW or NHWC.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output data tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Softplus",
    "schema": {
      "description": "\nSoftplus takes one input data tensor $X$ and produces one output data tensor $Y,$ where the softplus function, $y = ln(e^x + 1)$, is applied to $X$ elementwise.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/softplus_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/softplus_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Softplus\",\n    [\"X\"],\n    [\"Y\"],\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(3, 3).astype(np.float32))\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\n\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[-0.5380011   0.65190786  0.55673236]\n [-0.16272168  0.5451048   0.30880353]\n [-0.76606876 -0.6238556  -0.40444514]]\n\nY:\n [[0.4598992  1.0713093  1.0097669 ]\n [0.61509246 1.0023911  0.8594219 ]\n [0.38174385 0.42909983 0.5112337 ]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "Input data blob to be operated on.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob with same shape as input.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceL2",
    "schema": {
      "attributes": [
        {
          "description": "(*Tuple(int)*): list of axes to reduce",
          "name": "axes",
          "option": "optional"
        },
        {
          "description": "(*int*): set to 1 to keep the reduced dimension(s) (default=1), else set to 0 to not keep the reduced dimension(s)",
          "name": "keepdims",
          "option": "optional"
        }
      ],
      "description": "\nComputes the **L2 norm** of the input tensor's elements along the provided `axes`. The resulting tensor has the same rank as the input if the `keepdims` argument equals 1 (default). If `keepdims` is set to 0, then the `axes` dimensions are pruned.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduce_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ReduceL2\",\n    [\"X\"],\n    [\"Y\"],\n    axes=(0,1),\n    keepdims=0\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(1,2,5,5)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[[[ 8.  0.  2.  5.  1.]\n   [ 1.  3.  0.  4.  0.]\n   [ 1.  3.  6.  7.  7.]\n   [ 6.  9.  8.  4.  6.]\n   [ 6.  1.  5.  7.  3.]]\n\n  [[ 2.  4.  6.  2.  8.]\n   [ 1.  1.  8.  0.  8.]\n   [ 5.  9.  0.  3.  2.]\n   [ 1.  7.  3.  7.  3.]\n   [ 6.  8.  9.  8.  7.]]]]\n\nY:\n[[  8.24621105   4.           6.3245554    5.38516474   8.06225777]\n [  1.41421354   3.1622777    8.           4.           8.        ]\n [  5.09901953   9.48683262   6.           7.6157732    7.28010988]\n [  6.08276272  11.40175438   8.54400349   8.06225777   6.70820379]\n [  8.48528099   8.06225777  10.29563046  10.63014603   7.6157732 ]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): reduced tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "WallClockTime",
    "schema": {
      "description": "Time since epoch in nanoseconds.",
      "outputs": [
        {
          "description": "The time in nanoseconds.",
          "name": "time"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Cos",
    "schema": {
      "description": "\nCalculates the cosine of the given input tensor, element-wise.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/cos_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Cos\",\n    [\"X\"],\n    [\"Y\"]\n)\n\nworkspace.FeedBlob(\"X\", np.random.rand(5).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX: [0.6816719  0.76771533 0.933932   0.01404487 0.11862425]\nY: [0.7765203  0.71949923 0.5946774  0.99990135 0.9929724 ]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor calculated as the cosine of the input tensor, element-wise.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AveragePool1D",
    "schema": {
      "description": "AveragePool1D \nconsumes an input blob and applies average pooling across the the blob according\nto kernel sizes, stride sizes, pad lengths and dilation. Average pooling consists\nof taking the average value of a subset of the input tensor according to the kernel\nsize and downsampling the data into the output blob for further processing. The\n`brew` module has a wrapper for this operator for use in a `ModelHelper` object.\n\nPooling layers reduce the spatial dimensionality of the input blob. Each of the\noutput blob's dimensions will reduce according to:\n\n$$dim_{out}=\\frac{dim_{in}-kernel+2*pad}{stride}+1$$\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"AveragePool\",\n    [\"X\"],\n    [\"Y\"],\n    kernel=2,\n    stride=2,\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(1, 1, 6, 6).astype(np.float32)) // NCHW\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX:\n [[[[-0.2883434   0.43498734  0.05417408  1.912558    0.09390241\n    -0.33173105]\n   [ 1.633709    1.2047161   0.36964908  0.99961185  0.4184147\n     0.9989975 ]\n   [ 1.7644193   0.1789665   1.5812988  -0.6038542  -0.36090398\n     0.33195344]\n   [ 0.9457722  -0.95174325 -0.78124577  1.2062047   1.1903144\n     0.2586746 ]\n   [ 1.252104    0.32645547  1.8073524  -0.78397465  0.9978303\n    -0.97614396]\n   [ 0.5440196   1.5778259  -0.76750124  0.5051756   0.8838398\n    -0.37085298]]]]\n\nY:\n [[[[0.7462672  0.83399826 0.2948959 ]\n   [0.4843537  0.3506009  0.35500962]\n   [0.9251013  0.19026303 0.13366827]]]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor of shape NCHW or NHWC.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output data tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Col2Im",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsPositionalWeightedSum",
    "schema": {
      "description": "\nVariation of SparseLengthsWeightedSum operator, where, for each row,\nweights are accessed by indices [0..L-1], where L is the length of given row.\nThis is basically a fused operator of LengthsRangeFill + Gather +\nSparseWeightedSum\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToRowwiseQuantized8Bits",
          "name": "DATA"
        },
        {
          "description": "Scalar multipliers for the input slices. Must be a vector with the length matching the length of DATA",
          "name": "WEIGHT"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Size",
    "schema": {
      "description": "\nReturn a 1D tensor of type *int64* that contains the number of elements of the input tensor.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/utility_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Size\",\n    [\"X\"],\n    [\"size\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.randint(10, size=(3,3))))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"size:\", workspace.FetchBlob(\"size\"))\n\nworkspace.ResetWorkspace()\n\nworkspace.FeedBlob(\"X\", (np.random.rand(6,4)))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"size:\", workspace.FetchBlob(\"size\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[3 7 0]\n [0 1 6]\n [5 0 8]]\nsize: 9\nX:\n[[0.92017884 0.32115368 0.68692035 0.64135016]\n [0.8723328  0.77830265 0.80688656 0.25524236]\n [0.37970216 0.76407047 0.85689564 0.30692883]\n [0.69352573 0.42531502 0.16415212 0.59209324]\n [0.52684188 0.37094846 0.60670079 0.6489272 ]\n [0.94715906 0.34800557 0.61898769 0.28947359]]\nsize: 24\n\n```\n\n</details>\n\n      ",
      "inputs": [
        {
          "description": "*(type: Tensor)* Input tensor to calculate number of elements.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor)* 1D tensor of type int64 that contains the number of elements in the input tensor *X*.",
          "name": "size"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Fused8BitRowwiseQuantizedToFloat",
    "schema": {
      "description": "\nDe-quantizes the result of the\nFloatToFused8BitRowwiseQuantized operator. The input is expected to\nencode the scale as a 32-bit float in the second to the last 4 bytes of each\nrow, followed by the bias as a 32-bit float in the next 4 bytes, and the\nquantized values in the preceding bytes of the row. The output is a\nmatrix containing only the values, but de-quantized. De-quantization is\nperformed by multiplying each value by its row's scale and bias\nparameters. The de-quantized values will thus not be exactly equal to\nthe original, un-quantized floating point values.\n",
      "inputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "scale_bias_quantized_input"
        }
      ],
      "outputs": [
        {
          "description": "Float32 data",
          "name": "float_output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "NanCheck",
    "schema": {
      "description": "Identity operator, but checks all values for nan or inf",
      "inputs": [
        {
          "description": "Tensor to check for nan/inf",
          "name": "tensor"
        }
      ],
      "outputs": [
        {
          "description": "Tensor to copy input into if no NaNs or inf. Can be in-place",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CbrtGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "RowWiseSparseAdagrad",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\nGiven inputs (param, moment, indices, grad, lr), runs a modified sparse Adagrad\nupdate on (param, grad, moment[indices], lr), and returns (new_param,\nnew_momwnr), where moment is a 1D tensor with length equal to the number of\nrows in param: shape(moment) == shape(param)[0]. Each element of moment is\napplied to an entire row of param, and the new moment is calculated by adding\nthe average squared sum of gradients across each row. Note that indices must\nalso be a 1D tensor indexing into the rows of param.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Moment history",
          "name": "moment"
        },
        {
          "description": "Sparse indices",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated moment",
          "name": "output_moment_1"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GivenTensorDoubleFill",
    "schema": {
      "attributes": [
        {
          "description": "The value for the elements of the output tensor.",
          "name": "values"
        },
        {
          "description": "The shape of the output tensor.Cannot set the shape argument and pass in an input at the same time.",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "The additional dimensions appended at the end of the shape indicatedby the input blob.Cannot set the extra_shape argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional"
        },
        {
          "description": "1D tensor containing the desired output shape. First input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LengthsToRanges",
    "schema": {
      "description": "\nGiven a vector of segment lengths, calculates offsets of each segment and packs\nthem next to the lengths. For the input vector of length N the output is a Nx2\nmatrix with (offset, lengths) packaged for each segment.\n\nFor example, `[1, 3, 0, 2]` transforms into `[[0, 1], [1, 3], [4, 0], [4, 2]]`.\n",
      "inputs": [
        {
          "description": "1D tensor of int32 segment lengths.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "2D tensor of shape len(lengths) X 2 and the same type as `lengths`",
          "name": "ranges"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseSortedSegmentWeightedSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CosineEmbeddingCriterion",
    "schema": {
      "description": "\nCosineEmbeddingCriterion takes two inputs: the similarity value and\nthe label, and computes the elementwise criterion output as\n\n  output = 1 - s,               if y == 1\n           max(0, s - margin),  if y == -1\n",
      "inputs": [
        {
          "description": "The cosine similarity as a 1-dim TensorCPU.",
          "name": "S"
        },
        {
          "description": "The label as a 1-dim TensorCPU with int value of 1 or -1.",
          "name": "Y"
        }
      ],
      "outputs": [
        {
          "description": "The output loss with the same dimensionality as S.",
          "name": "loss"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "IsEmpty",
    "schema": {
      "description": "\nThe *IsEmpty* op accepts a single input $tensor$, and produces a single boolean output $is\\_empty$. The output is *True* if and only if $tensor$ has size == 0.\n\nGithub Links:\n\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/utility_ops.cc\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/utility_ops.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"IsEmpty\",\n    [\"tensor\"],\n    [\"is_empty\"],\n)\n\n// Use a not-empty tensor\nworkspace.FeedBlob(\"tensor\", np.random.randn(2, 2).astype(np.float32))\nprint(\"tensor:\\n\", workspace.FetchBlob(\"tensor\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"is_empty: \", workspace.FetchBlob(\"is_empty\"),\"\\n\")\n\n// Use an empty tensor\nworkspace.FeedBlob(\"tensor\", np.empty(0))\nprint(\"tensor:\\n\", workspace.FetchBlob(\"tensor\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"is_empty: \", workspace.FetchBlob(\"is_empty\"))\n\n```\n\n**Result**\n\n```\n\ntensor:\n [[ 0.26018378  0.6778789 ]\n [-1.3097627  -0.40083608]]\nis_empty:  False\n\ntensor:\n []\nis_empty:  True\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "Input data tensor to check if empty.",
          "name": "tensor"
        }
      ],
      "outputs": [
        {
          "description": "Output scalar boolean tensor. True if input has size == 0.",
          "name": "is_empty"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "KeySplit",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LC1D",
    "schema": {
      "description": "\nThe locally connected operator consumes an input vector, a 1D filter blob\nand a bias blob and computes the output. \nNote that other parameters, such as the stride and\nkernel size, or the pads' sizes in each direction are not necessary for input\nbecause they are provided by the ConvPoolOpBase operator. Various dimension\nchecks are done implicitly, and the sizes are specified in the Input docs for\nthis operator. As is expected, the filter is locally connected with a subset of\nthe image and the bias is added; this is done throughout the image data and the\noutput is computed. As a side note on the implementation layout:\nlocally_connected_op_impl.h is the templated implementation of the\nlocally_connected_op.h file, which is why they are separate files.\n",
      "inputs": [
        {
          "description": null,
          "name": null
        },
        {
          "description": "The filter blob that will be used in the locally connected op; has size (YH * YW * M x C x kH x kW) if order == NCHW else (YH * YW * M  * KH * KW * C), where YH and YW are the height and width of the output image, C is the number of channels, and kH and kW are the height and width of the kernel.",
          "name": "filter"
        },
        {
          "description": "The 1D bias blob that is added through the locally connected op; has size (YH * YW * M).",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob that contains the result of the locally connected op.The output dimensions are functions of the kernel size, stride size, and pad lengths.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GenerateProposalsCPP",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "XavierFill",
    "schema": {
      "attributes": [
        {
          "description": "Desired shape of the *output* tensor.",
          "name": "shape",
          "option": "optional",
          "type": "int64[]"
        },
        {
          "description": "The additional dimensions appended at the end of the *shape* indicated by the input blob. Cannot set the *extra_shape* argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional",
          "type": "int64[]"
        },
        {
          "default": false,
          "description": "set to *True* to use the *input* as shape. First, input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional",
          "type": "boolean"
        }
      ],
      "description": "\nThis op fills an output tensor with values sampled from a uniform distribution with the range determined by the desired shape of the output. Rather, than specifying the range of values manually, the novelty of Xavier Fill is that it automatically scales the range of the distribution it draws from based on the size of the desired output tensor. For more information check out the paper [Understanding the difficulty of training deep feedforward neural networks](http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf). The output tensor shape is specified by the *shape* argument. However, if *input_as_shape* is set to *true*, then the *input* should be a 1D tensor containing the desired output shape (the dimensions specified in *extra_shape* will also be appended). In this case, the *shape* argument should **not** be set.\n\n*Note: Do not set the shape argument and pass in an input at the same time.*\n\nGithub Links:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/filler_op.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/filler_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"XavierFill\",\n    [],\n    [\"out\"],\n    shape=[3,3],\n)\n\nworkspace.RunOperatorOnce(op)\nprint(\"Out:\\n\", workspace.FetchBlob(\"out\"))\n\n```\n\n**Result**\n\n```\n\nOut:\n [[-0.8412168   0.33207083 -0.88418937]\n [ 0.43059897 -0.8340702   0.07781601]\n [ 0.93261135 -0.24542928 -0.3980782 ]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(Optional) 1D tensor specifying the shape of the output. Must be used with *input_as_shape=True*",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor of random values drawn from an automatically scaled uniform distribution, based on the size of the output tensor. If the shape argument is set, this is the shape specified by the shape argument, and if the *input* exists and *input_as_shape=True*, it is the shape specified by the *input* tensor.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "QuantDecode",
    "schema": {
      "description": "\nDecode inputs using codebook. This is a general LUT operator that returns\ntensors with values from codebook (input 0) based on given indices in\ncodes (input 1 ~ n).\n\n\nExample:\n\n\nInput:\n  codebook = [1.5, 2.5, 3.5]\n  codes_0 = [0, 1, 1, 2]\n  codes_1 = [2, 0, 0]\n\n\nOutput:\n  decoded_0 = [1.5, 2.5, 2.5, 3.5]\n  decoded_1 = [3.5, 1.5, 1.5]\n",
      "inputs": [
        {
          "description": "Codebook in 1d tensor (float)",
          "name": "codebook"
        },
        {
          "description": "Encoded codes 0 (uint8/uint16/int32)",
          "name": "codes_0"
        },
        {
          "description": "Encoded codes 1 if existed (uint8/uint16/int32)",
          "name": "codes_1"
        },
        {
          "description": "Encoded codes n if existed (uint8/uint16/int32)",
          "name": "codes_n"
        }
      ],
      "outputs": [
        {
          "description": "Decoded tensor for codes_0 (float)",
          "name": "decoded_0"
        },
        {
          "description": "Decoded tensor for codes_1 (float)",
          "name": "decoded_1"
        },
        {
          "description": "Decoded tensor for codes_n (float)",
          "name": "decoded_n"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ElementwiseLinearGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "TimerGetAndEnd",
    "schema": {
      "description": "\nQueries the current time of a timer in nanos, stops the timer publishing a CAFFE_EVENT.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/stats_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\ntimerbegin_op = core.CreateOperator(\n    \"TimerBegin\",\n    [],\n    [\"timer\"]\n)\n\ntimerget_op = core.CreateOperator(\n    \"TimerGet\",\n    [\"timer\"],\n    [\"nanos\"]\n)\n\ntimerend_op = core.CreateOperator(\n    \"TimerEnd\",\n    [\"timer\"],\n    []\n)\n\ntimergetandend_op = core.CreateOperator(\n    \"TimerGetAndEnd\",\n    [\"timer\"],\n    [\"nanos\"]\n)\n\n// Test TimerBegin/TimerGet/TimerEnd\nworkspace.RunOperatorOnce(timerbegin_op)\nprint(\"timer:\", workspace.FetchBlob(\"timer\"))\nworkspace.RunOperatorOnce(timerget_op)\nprint(\"nanos:\", workspace.FetchBlob(\"nanos\"))\nworkspace.RunOperatorOnce(timerend_op)\n\n\n// Test TimerBegin/TimerGetAndEnd\nworkspace.RunOperatorOnce(timerbegin_op)\nprint(\"timer:\", workspace.FetchBlob(\"timer\"))\nworkspace.RunOperatorOnce(timergetandend_op)\nprint(\"nanos:\", workspace.FetchBlob(\"nanos\"))\n\n```\n\n**Result**\n\n```\n\ntimer: b'timer, a C++ native class of type caffe2::TimerInstance*.'\nnanos: 361140\ntimer: b'timer, a C++ native class of type caffe2::TimerInstance*.'\nnanos: [252250]\n\n```\n\n</details>\n\n      ",
      "inputs": [
        {
          "description": "(*Tensor`<ptr>`*): pointer to a timer object; obtained from **TimerBegin** op",
          "name": "timer"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<int64>`*): scalar tensor containing time in nanoseconds",
          "name": "nanos"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "DivGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ReduceMaxGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Do",
    "schema": {
      "attributes": [
        {
          "description": "Subnet with blob bindings",
          "name": "net",
          "option": "optional"
        },
        {
          "description": "List of inner net blob names to bind to outer workspace",
          "name": "inner_blobs",
          "option": "optional"
        },
        {
          "description": "Indices of corresponding outer workspace blobs, in order: operator inputs, operator outputs (skipping workspace blobs)",
          "name": "outer_blobs_idx",
          "option": "optional"
        },
        {
          "description": "List of blobs from the forward Do operator workspace needed in backward pass, used in gradient Do operator",
          "name": "saved_fwd_blobs",
          "option": "optional"
        },
        {
          "description": "Whether to reuse workspace or create a new one in a given scope",
          "name": "reuse_workspace",
          "option": "optional"
        }
      ],
      "description": "\n'Do' control operator, executes a subnet in a separate workspace.\nLast blobs in the input and output lists should be the same blob created with\nCreateScope op. Arguments 'inner_blobs' and 'outer_blobs_idx' provide a mapping\nbetween selected inner blob names and corresponding outer blob indices.\n    ",
      "support_level": "default"
    }
  },
  {
    "name": "DotProductWithPaddingGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "UniqueUniformFill",
    "schema": {
      "attributes": [
        {
          "description": "Minimum value, inclusive",
          "name": "min",
          "option": "optional"
        },
        {
          "description": "Maximum value, inclusive",
          "name": "max",
          "option": "optional"
        },
        {
          "description": "The data type for the elements of the output tensor.Strictly must be one of the types from DataType enum in TensorProto.This only supports INT32 and INT64 now. If not set, assume INT32",
          "name": "dtype",
          "option": "optional"
        },
        {
          "description": "The shape of the output tensor.Cannot set the shape argument and pass in an input at the same time.",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "The additional dimensions appended at the end of the shape indicatedby the input blob. Cannot set the extra_shape argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional"
        },
        {
          "description": "1D tensor containing the desired output shape. First input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": "\nFill the output tensor with uniform samples between min and max (inclusive).\nIf the second input is given, its elements will be excluded from uniform\nsampling. Using the second input will require you to provide shape via the first\ninput.\n",
      "inputs": [
        {
          "description": "Input tensor to provide shape information",
          "name": "input"
        },
        {
          "description": "(optional) Avoid elements in this tensor. Elements must be unique.",
          "name": "avoid"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor of unique uniform samples",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LongIndexCreate",
    "schema": {
      "attributes": [
        {
          "description": "Max number of elements, including the zero entry.",
          "name": "max_elements",
          "option": "optional"
        }
      ],
      "description": "\nCreates a dictionary that maps int64 keys to consecutive integers\nfrom 1 to max_elements. Zero is reserved for unknown keys.\n",
      "outputs": [
        {
          "description": "Pointer to an Index instance.",
          "name": "handler"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ComputeOffset",
    "schema": {
      "description": "\nCompute the offsets matrix given cursor and data blobs. Need to be ran at\nbeginning or after reseting cursor\n\nInput(0) is a blob pointing to a TreeCursor, and\n[Input(1),... Input(num_fields)] a list of tensors containing the data for\neach field of the dataset.\n\nComputeOffset is thread safe.\n",
      "inputs": [
        {
          "description": "A blob containing a pointer to the cursor.",
          "name": "cursor"
        },
        {
          "description": "First dataset field",
          "name": "dataset_field_0"
        }
      ],
      "outputs": [
        {
          "description": "Tensor containing offset info for this chunk.",
          "name": "field_0"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ByteWeightDequant",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CopyOnDeviceLike",
    "schema": {
      "description": "Copy input tensor into output to the specific device.",
      "inputs": [
        {
          "description": "The input tensor.",
          "name": "input"
        },
        {
          "description": "Tensor, on which device the copy will be performed.",
          "name": "dst"
        }
      ],
      "outputs": [
        {
          "description": "Tensor that will contain a copy of the input.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BatchOneHot",
    "schema": {
      "description": "\nInput is a matrix tensor. Its first dimension is the batch\nsize. Expand each column of it using one hot encoding. The `lengths` specifies\nthe size of each column after encoding, and the `values` is the dictionary value\nof one-hot encoding for each column. For example\n\n  If data = [[2, 3], [4, 1], [2, 5]], lengths = [2, 3],\n  and values = [2, 4, 1, 3, 5], then\n\n  output = [[1, 0, 0, 1, 0], [0, 1, 1, 0, 0], [1, 0, 0, 0, 1]]\n",
      "inputs": [
        {
          "description": "input tensor matrix",
          "name": "data"
        },
        {
          "description": "the size is the same as the width of the `data`",
          "name": "lengths"
        },
        {
          "description": "one hot encoding dictionary values",
          "name": "values"
        }
      ],
      "outputs": [
        {
          "description": "output matrix that expands each input column with one hot encoding",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "DropoutGrad",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "MulGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "MarginRankingCriterionGradient",
    "schema": {
      "description": "\nMarginRankingCriterionGradient takes both X1, X2, Y and dY and\nuses them to update dX1, and dX2 according to the chain rule\nand derivatives of the loss function.\n",
      "support_level": "default"
    }
  },
  {
    "name": "CreateMutex",
    "schema": {
      "description": "Creates an unlocked mutex and returns it in a unique_ptr blob.",
      "outputs": [
        {
          "description": "Blob containing a std::unique_ptr<mutex>.",
          "name": "mutex_ptr"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Float16UniformFill",
    "schema": {
      "attributes": [
        {
          "description": "Shape of the tensor",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "Minimim value to generate",
          "name": "min",
          "option": "optional"
        },
        {
          "description": "Maximum value to generate",
          "name": "max",
          "option": "optional"
        }
      ],
      "description": "Fills a half float tensor of a specified shape with values from a uniform distribution[min,max]",
      "support_level": "default"
    }
  },
  {
    "name": "SparseAdadelta",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        },
        {
          "description": "Default 0.95, the squared gradient sum is decayed by this factor.",
          "name": "decay",
          "option": "optional"
        }
      ],
      "description": "\n\nGiven inputs (param, moment, moment_delta, indices, grad, lr),\nruns the dense AdaDelta update on (param, grad, moment[indices],\n moment_delta[indices], lr), and returns (new_param, new_moment,\n new_moment_delta) as in the dense case.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Average of squared gradients",
          "name": "moment"
        },
        {
          "description": "Average of squared parameter updates",
          "name": "moment_delta"
        },
        {
          "description": "Sparse indices",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated average squared gradient",
          "name": "output_moment"
        },
        {
          "description": "Updated average of squared parameter updates",
          "name": "output_moment_delta"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FloatToRowwiseQuantized8Bits",
    "schema": {
      "description": "\nThis operator applies 8Bit row-wise quantization to\ninput tensor and returns quantized tensor. Row wise quantization of\ninput tensor is the following process. We take tensor of size\n(m_1, m_2,...,m_n), n >= 2, reshape it into matrix of size\n(m_1, m_2 x... x m_n) and apply row-wise quantization. After this,\nwe compute scale_i= (min_i - max_i) / 255 and  bias_i = min_i for\ni-th row r_i of reshaped matrix, where min_i and max_i --  minimum\nand maximum elements of i-th row, and quantize each element r_{ij} as\n0 <= round(r_ij - bias_i) / scale_i) < 256. Instead of input tensor\nwe obtain uint8 tensor and auxiliary information as scale and bias to\nrestore input tensor (with losses).\n",
      "inputs": [
        {
          "description": "input",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "quantized_input",
          "name": "quantized_input"
        },
        {
          "description": "Matrix of floats, each row r_i of which stores a pair s_i, b_i",
          "name": "scale_bias"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SumRelu",
    "schema": {
      "description": null,
      "inputs": [
        {
          "description": "First of the input tensors. Can be inplace.",
          "name": "data_0"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor. Same dimension as inputs.",
          "name": "sum"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LSTMUnitGradient",
    "schema": {
      "attributes": [
        {
          "description": "When false, the sequence lengths input is left out, and all following inputs are shifted left by one.",
          "name": "sequence_lengths",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "AveragePool3DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentRangeMean",
    "schema": {
      "description": "\nApplies 'Mean' to each segment of input tensor. In order to allow for more\nefficient implementation of 'Mean', the input segments have to be contiguous\nand non-empty.\n\nSEGMENT_IDS is a vector that maps each of the first dimension slices of the\nDATA to a particular group (segment). Values belonging to the same segment are\naggregated together.\n\nThe first dimension of the output is equal to the number of input segments,\ni.e. `SEGMENT_IDS[-1]+1`. Other dimensions are inherited from the input tensor.\n\nMean computation is done element-wise, so that each element of the output slice corresponds to the average value of the respective elements in the input slices. Operation doesn't change the shape of individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor to be aggregated",
          "name": "DATA"
        },
        {
          "description": "Vector with the same length as the first dimension of DATA and values in the range 0..K-1 and in increasing order that maps each slice of DATA to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated tensor with the first dimension of K and the other dimentsions inherited from DATA",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "DiagonalFill",
    "schema": {
      "attributes": [
        {
          "description": "The value for the elements of the output tensor.",
          "name": "value",
          "option": "optional"
        },
        {
          "description": "The data type for the elements of the output tensor.Strictly must be one of the types from DataType enum in TensorProto.",
          "name": "dtype",
          "option": "optional"
        },
        {
          "description": "The shape of the output tensor.Cannot set the shape argument and pass in an input at the same time.",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "The additional dimensions appended at the end of the shape indicatedby the input blob.Cannot set the extra_shape argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional"
        },
        {
          "description": "1D tensor containing the desired output shape",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": "\nThe operator fills the diagonal elements of the output tensor (>= 2D)\nwith a constant value specified by the 'value' argument, and others 0. If\nnumber of dimensions of the output tensor is greater than 2, all dimensions\nmust be equal.\n\nThe data type is specified by the 'dtype' argument. The 'dtype' argument must\nbe one of the data types specified in the 'DataType' enum field in the\nTensorProto message. If the 'dtype' argument is not provided, the data type of\n'value' is used.\n\nThe output tensor shape is specified by the 'shape' argument. If the number of\ninput is 1, the shape will be identical to that of the input at run time with\noptional additional dimensions appended at the end as specified by 'extra_shape'\nargument. In that case the 'shape' argument should not be set.\n\nIf input_as_shape is set to true, then the input should be a 1D tensor\ncontaining the desired output shape (the dimensions specified in extra_shape\nwill also be appended)\n\nNOTE: Currently, it supports data type of float, int32, int64, and bool.\n",
      "inputs": [
        {
          "description": "Input tensor (optional) to provide shape information.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Output tensorargument and its type is specified by the 'dtype' argument",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ConcatTensorVector",
    "schema": {
      "description": "\nConcat Tensors in the std::unique_ptr<std::vector<Tensor> >\nalong the first dimension.\n    ",
      "inputs": [
        {
          "description": "std::unique_ptr<std::vector<Tensor> >",
          "name": "vector of Tensor"
        }
      ],
      "outputs": [
        {
          "description": "tensor after concatenating",
          "name": "tensor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Conv2D",
    "schema": {
      "description": "\nThe convolution operator consumes an input vector, a 2D filter blob\nand a bias blob and computes the output. \nThe Conv2D operator computes a 2D convolution operation over an input blob $(X)$, with a filter blob $(filter)$ and a bias blob $(bias)$, and outputs a single output blob $(Y)$. Although there are several options for order, the convention is that the input $(X)$ is a blob of shape $(N,C_{in},H_{in},W_{in})$ and the output $(Y)$ is a blob of shape $(N,C_{out},H_{out},W_{out})$. Here, $N$ is the batch size, $C$ is the number of channels, $H$ is the spatial height, and $W$ is the spatial width. For example, if your input data was a batch of five, 100x120pixel RGB images, $X$ would have shape $(5,3,120,100)$.\n\nThe $filter$ input blob may contain multiple filters and has shape $(M, C_{in}, K_H, K_W)$. Here, $M$ is the number of individual filters contained in the blob, $C_{in}$ is the number of channels of each filter (by convention in 2D convolution it is the same as the number of channels in the input), $K_H$ is the spatial height of the kernel, and $K_W$ is the spatial width of the kernel. The $bias$ blob is a vector of length $M$, where there is one bias for each filter in the $filter$ blob.\n\nGiven the shape of the input blob and the filter blob, we can calculate the shape of the output blob as follows. The number of items in the batch $N$ will stay the same. The number of channels in the output will equal the number of kernels in the filter blob, so $C_{out} = M.$ With stride and pad defined below, the spatial height and width of the output ($H_{out}$ and $W_{out}$) are calculated as\n\n$$H_{out} = \\left \\lfloor{\\frac{H_{in} - K_H + 2*pad}{stride}+1}\\right \\rfloor$$\n\n\n$$W_{out} = \\left \\lfloor{\\frac{W_{in} - K_W + 2*pad}{stride}+1}\\right \\rfloor$$\n\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Conv\",\n    [\"X\", \"filter\", \"bias\"],\n    [\"Y\"],\n    kernel=5,\n    pad=1,\n    stride=2\n)\n\n// Create X: (N,C,H,W)\ndata = np.random.randn(1,1,8,8).astype(np.float32)\nprint(\"Data shape: \",data.shape)\n\n// Create W: (M,C,Kh,Kw)\nfilters = np.random.randn(3,1,5,5).astype(np.float32)\nprint(\"Filter shape: \",filters.shape)\n\n// Create b: M\nbias = np.array([1.,1.,1.]).astype(np.float32)\nprint(\"Bias shape: \",bias.shape)\n\n// Put the inputs into the workspace\nworkspace.FeedBlob(\"X\", data)\nworkspace.FeedBlob(\"filter\", filters)\nworkspace.FeedBlob(\"bias\", bias)\n\n// Run the operator\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nData shape:  (1, 1, 8, 8)\nFilter shape:  (3, 1, 5, 5)\nBias shape:  (3,)\nY:\n [[[[  0.6406407    0.8620521    0.56461596]\n   [ -1.5042953   -0.79549205 -10.683343  ]\n   [ -0.5240259    3.4538248   -3.9564204 ]]\n\n  [[  0.6876496    4.8328524   -1.9525816 ]\n   [  1.2995434   -2.3895378    7.2670045 ]\n   [  3.9929862    1.8126237    5.4699917 ]]\n\n  [[  3.55949      4.7934155    0.76086235]\n   [  3.9588015   -1.3251319    4.413117  ]\n   [ -1.5296054   -1.4924102   -3.2552304 ]]]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "Input data blob, of shape $(N, C_{in}, H_{in}, W_{in})$, to be convolved with the kernels in the filter blob.",
          "name": "X"
        },
        {
          "description": "The filter blob, of shape $(M, C_{in}, K_H, K_W)$, containing the filters to be convolved with the data.",
          "name": "filter"
        },
        {
          "description": "The bias blob, of length $M$, containing the biases for the convolution, one bias per filter.",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob, of shape $(N, C_{out}, H_{out}, W_{out})$, that contains the result of the convolution.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MultiClassAccuracy",
    "schema": {
      "description": "\nRespectively compute accuracy score for each class given a number of instances\nand predicted scores of each class for each instance.\n",
      "inputs": [
        {
          "description": "2-D float tensor (N,D,) of predicted scores of each class for each data. N is the number of instances, i.e., batch size. D is number of possible classes/labels.",
          "name": "prediction"
        },
        {
          "description": "1-D int tensor (N,) of labels for each instance.",
          "name": "labels"
        }
      ],
      "outputs": [
        {
          "description": "1-D float tensor (D,) of accuracy for each class. If a class has no instance in the batch, its accuracy score is set to zero.",
          "name": "accuracies"
        },
        {
          "description": "1-D int tensor (D,) of number of instances for each class in the batch.",
          "name": "amounts"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceL2Gradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseUnsortedSegmentSum",
    "schema": {
      "description": "\nPulls in slices of the input tensor, groups them into segments and applies\n'Sum' to each segment. Segments ids can appear in arbitrary order (unlike in\nSparseSortedSegmentSum).\n\nThis op is basically Gather and UnsortedSegmentSum fused together.\n\nINDICES should contain integers in range 0..N-1 where N is the first dimension\nof DATA. INDICES represent which slices of DATA need to be pulled in.\n\nSEGMENT_IDS is a vector that maps each referenced slice of the DATA to a\nparticular group (segment). Values belonging to the same segment are aggregated\ntogether. SEGMENT_IDS should have the same dimension as INDICES.\n\nIf `num_segments` argument is passed it would be used as a first dimension for\nthe output. Otherwise, it'd be dynamically calculated from as the max value of\nSEGMENT_IDS plus one. Other output dimensions are inherited from the input\ntensor.\n\nSummation is done element-wise across slices of the input tensor and doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Integer vector with the same length as INDICES that maps each slice of DATA referenced by INDICES to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of equal to the number of segments.",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceFrontMax",
    "schema": {
      "attributes": [
        {
          "description": "(*int*): number of dimensions to reduce (default=1)",
          "name": "num_reduce_dims",
          "option": "optional"
        }
      ],
      "description": "\nReduces the input tensor along the last dimension of the by applying **max**.\n\nCan reduce more than one of the \"first\" dimensions by setting `num_reduce_dim`.\n\nA second (optional) input, `lengths`, can be passed, which enforces that only a subset of the elements are considered in the max operation.\n- If input tensor `X` has shape $(d_0, d_1, d_2, ..., d_n)$, `lengths` must have shape $(d_1 * d_2 * ... * d_{n})$.\n- The values of the `lengths` tensor determine how many of the values to consider for each vector in the $d_{0}$ dimension.\n\nFor example if $X = [[1,5,2,9],[4,1,8,2],[2,7,0,3]]$ and $lengths = [2,3,1,2]$, then $Y = [max(1,4), max(5,1,7), max(2), max(9,2)] = [4, 7, 2, 9]$\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduce_front_back_max_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ReduceFrontMax\",\n    [\"X\"],\n    [\"Y\"],\n    num_reduce_dim=2\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(2,3,3)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[[2. 8. 1.]\n  [9. 6. 6.]\n  [7. 7. 0.]]\n\n [[4. 3. 9.]\n  [9. 2. 7.]\n  [6. 4. 7.]]]\nY: [9. 8. 9.]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor",
          "name": "X"
        },
        {
          "description": "(*Tensor`<int>`*): number of elements in each sample",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): reduced tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ExpandDims",
    "schema": {
      "attributes": [
        {
          "description": "List of dimensions of *data* to add single dimensional entry.",
          "name": "dims",
          "option": "optional",
          "type": "int64[]"
        }
      ],
      "description": "\nThe *ExpandDims* op inserts single-dimensional entries into the shape of the input tensor *data,* and produces a single output tensor *expanded*. The op also takes an argument *dims* with a list of dimensions for where to add the single dimensional entries. If the same blob is provided as input and output, the operation is copy-free. This is the exact inverse operation of *Squeeze*.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/expand_squeeze_dims_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/expand_squeeze_dims_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ExpandDims\",\n    [\"data\"],\n    [\"expanded\"],\n    dims=[0,1],\n)\n\nworkspace.FeedBlob(\"data\", np.zeros((100,100)).astype(np.float32))\nprint(\"data.shape:\", workspace.FetchBlob(\"data\").shape)\n\nworkspace.RunOperatorOnce(op)\nprint(\"expanded.shape:\", workspace.FetchBlob(\"expanded\").shape)\n\n```\n\n**Result**\n\n```\n\ndata.shape: (100, 100)\nexpanded.shape: (1, 1, 100, 100)\n\n```\n\n</details>\n\n\n\n",
      "inputs": [
        {
          "description": "Input tensor of data to be operated on.",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "Reshaped tensor with same data as input.",
          "name": "expanded"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RowMul",
    "schema": {
      "description": "\nGiven a matrix A and column vector w, the output is the multiplication of row i\nof A and element i of w, e.g. C[i][j] = A[i][j] * w[i]. This operator should be\ndeprecated when the gradient operator of Mul with broadcast is implemented.\n",
      "inputs": [
        {
          "description": "The matrix",
          "name": "mat"
        },
        {
          "description": "The column vector",
          "name": "w"
        }
      ],
      "outputs": [
        {
          "description": "Output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BatchMoments",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "IsMemberOf",
    "schema": {
      "attributes": [
        {
          "description": "List of values to check for membership.",
          "name": "value",
          "option": "optional"
        },
        {
          "description": "The data type for the elements of the output tensor. Strictly must be one of the types from DataType enum in TensorProto.",
          "name": "dtype",
          "option": "optional"
        }
      ],
      "description": "\nThe *IsMemberOf* op takes an input tensor *X* and a list of values as argument, and produces one output data tensor *Y*. The output tensor is the same shape as *X* and contains booleans. The output is calculated as the function *f(x) = x in value* and is applied to *X* elementwise.\n\nGithub Links:\n\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/elementwise_logical_ops.cc\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/elementwise_logical_ops.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"IsMemberOf\",\n    [\"X\"],\n    [\"Y\"],\n    value=[0,2,4,6,8],\n)\n\n// Use a not-empty tensor\nworkspace.FeedBlob(\"X\", np.array([0,1,2,3,4,5,6,7,8]).astype(np.int32))\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"Y: \\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n// value=[0,2,4,6,8]\n\nX:\n [0 1 2 3 4 5 6 7 8]\nY:\n [ True False  True False  True False  True False  True]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "Input tensor of any shape",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor (same size as X containing booleans)",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MinGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "RangeFill",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ReluN",
    "schema": {
      "attributes": [
        {
          "description": "the cap of output",
          "name": "n",
          "option": "optional"
        }
      ],
      "description": "\nRelu takes one input data (Tensor) and produces one output data\n(Tensor) where the rectified linear function, y = min(max(0, x), n),\nis applied to the tensor elementwise.\n",
      "inputs": [
        {
          "description": "1D input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "1D input tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "TanhGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CubeGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ReduceTailSum",
    "schema": {
      "description": "\nReduce the tailing dimensions\n",
      "inputs": [
        {
          "description": "The matrix",
          "name": "mat"
        }
      ],
      "outputs": [
        {
          "description": "Output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GroupNormGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Moments",
    "schema": {
      "attributes": [
        {
          "description": "A list of integers, along which to reduce. If axes is not provided, the op computes the element-wise mean and variance.",
          "name": "axes",
          "option": "optional"
        },
        {
          "description": "Keep the reduced dimension(s) or not, default True keeps the reduced dimension(s).",
          "name": "keepdims",
          "option": "optional"
        }
      ],
      "description": "\n  Computes the mean and variance of the input tensor's element along the\n  provided axes. The resulted tensor has the same rank as the input if keepdims\n  equals True.\n  If keepdims equals False, then the resulted tensor have the reduced dimension\n  pruned.\n",
      "inputs": [
        {
          "description": "An input tensor.",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "Reduced mean tensor.",
          "name": "mean"
        },
        {
          "description": "Reduced variance tensor.",
          "name": "variance"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ATen",
    "schema": {
      "description": null,
      "support_level": "contribution"
    }
  },
  {
    "name": "LC1DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LengthsToSegmentIds",
    "schema": {
      "description": "\nGiven a vector of segment lengths (*lengths*) the *LengthsToSegmentIds* op returns a zero-based, consecutive vector of segment ids (*segment_ids*). For example, *lengths=[1, 3, 0, 2]* will produce *segment_ids=[0, 1, 1, 1, 3, 3]*. In general, the inverse operation is *SegmentIdsToLengths*. Notice though that trailing empty sequence lengths can't be properly recovered from segment ids.\n\nGithub Links:\n\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/utility_ops.cc\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/utility_ops.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LengthsToSegmentIds\",\n    [\"lengths\"],\n    [\"segment_ids\"],\n)\n\nworkspace.FeedBlob(\"lengths\", np.array([1, 3, 0, 2]).astype(np.int32))\nprint(\"lengths:\\n\", workspace.FetchBlob(\"lengths\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"segment_ids: \\n\", workspace.FetchBlob(\"segment_ids\"))\n\n```\n\n**Result**\n\n```\n\nlengths:\n [1 3 0 2]\nsegment_ids:\n [0 1 1 1 3 3]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "1D tensor of int32 or int64 segment lengths.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "1D tensor of length *sum(lengths)*",
          "name": "segment_ids"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Wngrad",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\nComputes the WnGrad update for an input gradient and accumulated\nhistory. This operator implement the optimization algorithm\nin https://arxiv.org/abs/1803.02865 by Wu, Ward and Bottou.\nConcretely, given inputs (param, grad, seq_b, learning_rate),\ncomputes\n\n    new_seq_b = seq_b + 1 / seq_b * norm(grad)^2\n    effective_lr = learning_rate / (new_seq_b + epsilon)\n    update = learning_rate * grad / (new_seq_b + epsilon)\n    new_param = param + update\nand returns (new_param, new_seq_b).\n\nOptionally returns effective_lr and update as well.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Seq_b history",
          "name": "seq_b"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated seq_b",
          "name": "output_seq_b"
        },
        {
          "description": "(optional) Effective learning rate",
          "name": "output_effective_lr"
        },
        {
          "description": "(optional) Actual update that is applied.",
          "name": "output_update"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Or",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise logical operation **or** (with limited broadcast support).\nBoth input operands should be of type `bool`.\n\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Or\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", (np.random.rand(3, 3) > 0.5))\nworkspace.FeedBlob(\"B\", (np.random.rand(3, 3) > 0.5))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\n\nA:\n[[False  True  True]\n [False  True  True]\n [ True  True  True]]\nB:\n[[False  True False]\n [ True  True  True]\n [False  True False]]\nC:\n[[False  True  True]\n [ True  True  True]\n [ True  True  True]]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "*(type: Tensor`<bool>`)* First operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<bool>`)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<bool>`)* Output tensor of booleans. Has same dimensions as input `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "EQ",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise equal to comparison **==** (with limited broadcast support).\n\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"EQ\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", np.array([1, 5, 2, 9, 12, 3]))\nworkspace.FeedBlob(\"B\", np.array([1, 3, 4, 9, 12, 8]))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\nA: [ 1  5  2  9 12  3]\nB: [ 1  3  4  9 12  8]\nC: [ True False False  True  True False]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<bool>`)* First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<bool>`)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<bool>`)* Output tensor with same dimensions as `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ErfGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ChannelBackpropStats",
    "schema": {
      "description": "\nGiven an input tensor in NCHW format, the gradient for the output of SpatialBN\nand the per-channel mean and inverse std var vectors for the input, computes the\nper-channel bias and scale gradient to be used during the backward pass for\nsubsequent spatial batch normalization gradient calculation. Typically, the\nresults of this op are subsequently reduced over multiple devices to obtain\nstatistics over a larger batch size in cases where the batch size for a single\nmodel copy is too low to yield the full benefit of batch normalization. The\nresulting bias and scale can then be plugged back into SpatialBNGradient to get\nresults over the larger batch size ",
      "inputs": [
        {
          "description": "The input 4-dimensional tensor of shape NCHW",
          "name": "X"
        },
        {
          "description": "The mean saved from the forward pass as a 1-dimensional tensor of size C.",
          "name": "mean"
        },
        {
          "description": "The saved inverse standard deviation as a 1-dimensional tensor of size C.",
          "name": "inv_std"
        },
        {
          "description": "Gradient for the output layer of SpatialBN, here used as input because we are on the backward pass",
          "name": "output_grad"
        }
      ],
      "outputs": [
        {
          "description": "Gradient for the scale vector",
          "name": "scale_grad"
        },
        {
          "description": "Gradient for the bias vector",
          "name": "bias_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GatherRangesToDense",
    "schema": {
      "attributes": [
        {
          "description": "Expected lengths for ranges",
          "name": "lengths",
          "option": "optional"
        },
        {
          "description": "The number of observations needed before deciding that the ratio of mismatched ranges is alarming, also determines whether an info sumarizing the empty and mismatch ratio will be printed at the end.",
          "name": "min_observation",
          "option": "optional"
        },
        {
          "description": "An error is raised when ratio of empty ranges exceeds this (default is 1, which means by default no error will be triggered).",
          "name": "max_empty_ratio",
          "option": "optional"
        },
        {
          "description": "An error is raised when ratio of mismatched ranges exceeds this.",
          "name": "max_mismatched_ratio",
          "option": "optional"
        },
        {
          "description": "A log is recorded only after an error is triggered every n times.",
          "name": "log_every_n",
          "option": "optional"
        }
      ],
      "description": "\nGiven DATA tensor of rank 1, and RANGES tensor of rank 3, gather values\ncorresponding to each range into a separate output tensor. If the optional input\nKEY tensor is also given, the output will be sorted by KEY for each example.\n\nRANGES dimensions description:\n1: represents list of examples within a batch\n2: represents list features\n3: two values which are start and length or a range (to be applied on DATA)\n\nEach feature has fixed lengths which are passed as lengths argument and a\nseparate tensor will be produced for each feature.\ni.e. DATA.dim(1) = len(lengths) = NumOuptuts.\n\nMissing features (represented by empty ranges) filled with default_value.\n\nExample 1:\n  DATA  = [1, 2, 3, 4, 5, 6, 7, 8]\n  RANGES = [\n    [\n      [2, 4],\n      [0, 2],\n    ],\n    [\n      [0, 0],\n      [6, 2],\n    ]\n  ]\n  lengths = [4, 2]\n  OUTPUT[0] = [[3, 4, 5, 6], [0, 0, 0, 0]]\n  OUTPUT[1] = [[1, 2], [7, 8]]\n\nExample 2 (with KEY):\nDATA  = [1, 2, 3, 4, 5, 6, 7, 8]\nKEY   = [0, 1, 3, 2, 1, 0, 1, 0]\nRANGES = [\n  [\n    [2, 4],\n    [0, 2],\n  ],\n  [\n    [0, 0],\n    [6, 2],\n  ]\n]\nlengths = [4, 2]\nOUTPUT[0] = [[6, 5, 4, 3], [0, 0, 0, 0]]\nOUTPUT[1] = [[1, 2], [8, 7]]\n\nContrast Example 2 with Example 1. For each data point per feature, the values\nare sorted by the corresponding KEY.\n",
      "inputs": [
        {
          "description": "Tensor of rank 1.",
          "name": "DATA"
        },
        {
          "description": "Tensor of int32/int64 ranges, of dims (N, M, 2). Where N is number of examples and M is a size of each example. Last dimension represents a range in the format (start, lengths)",
          "name": "RANGES"
        },
        {
          "description": "Tensor of rank 1 and type int64.",
          "name": "KEY"
        }
      ],
      "outputs": [
        {
          "description": "1-D tensor of size sum of range lengths",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LambdaRankNdcg",
    "schema": {
      "description": "\nIt implements the LambdaRank as appeared in Wu, Qiang, et al. \"Adapting boosting\nfor information retrieval measures.\" Information Retrieval 13.3 (2010): 254-270.\n\nThis method heuristically optimizes the NDCG.\n",
      "support_level": "default"
    }
  },
  {
    "name": "TileGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ResetCursor",
    "schema": {
      "description": "\nResets the offsets for the given TreeCursor. This operation is thread safe.\n",
      "inputs": [
        {
          "description": "A blob containing a pointer to the cursor.",
          "name": "cursor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentRangeMax",
    "schema": {
      "description": "\nApplies 'Max' to each segment of input tensor. In order to allow for more\nefficient implementation of 'Max', the input segments have to be contiguous\nand non-empty.\n\nSEGMENT_IDS is a vector that maps each of the first dimension slices of the\nDATA to a particular group (segment). Values belonging to the same segment are\naggregated together.\n\nThe first dimension of the output is equal to the number of input segments,\ni.e. `SEGMENT_IDS[-1]+1`. Other dimensions are inherited from the input tensor.\n\nMax computation is done element-wise, so that each element of the output slice corresponds to the max value of the respective elements in the input slices. Operation doesn't change the shape of individual blocks. This implementation imitates torch nn.Max operator. If the maximum value occurs more than once, the operator will return the first occurrence of value. When computing the gradient using the backward propagation, the gradient input corresponding to the first occurrence of the maximum value will be used.\n  ",
      "inputs": [
        {
          "description": "Input tensor to be aggregated",
          "name": "DATA"
        },
        {
          "description": "Vector with the same length as the first dimension of DATA and values in the range 0..K-1 and in increasing order that maps each slice of DATA to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated tensor with the first dimension of K and the other dimentsions inherited from DATA",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PairWiseLoss",
    "schema": {
      "description": "\nOperator computes the pair wise loss between all pairs within a batch\n using the logit loss function on the difference in scores between pairs\n",
      "inputs": [
        {
          "description": "Input blob from the previous layer, which is almost always the result of a softmax operation; X is a 2D array of size N x 1where N is the batch size. For more info: D. Sculley, Large Scale Learning to Rank. https://www.eecs.tufts.edu/~dsculley/papers/large-scale-rank.pdf",
          "name": "X"
        },
        {
          "description": "Blob containing the labels used to compare the input",
          "name": "label"
        },
        {
          "description": "Optional input blob that contains the lengthsof multiple sessions. The summation of this blob must be equalto the size of blob X. If lengths blob is provided, the outputblob has the same size as lengths blob, and the cross entropyis computed within each session.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Output blob after the cross entropy computation",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "NHWC2NCHW",
    "schema": {
      "description": "\nThe operator switches the order of data in a tensor from NHWC- sample index N,\nheight H, width H and channels C, to the NCHW order (this is for 2D images).\nIn general, this operator switches the order of data in a tensor from N H_1 ...\nH_k C to N C H_1 ... H_k for k-dimensional features, and currently supports\nk=1, 2, and 3.\n",
      "inputs": [
        {
          "description": "The input data (Tensor) in the NHWC order.",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "The output tensor (Tensor) in the NCHW order.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CreateRebatchingQueue",
    "schema": {
      "attributes": [
        {
          "description": "Number of input tensors the queue will support",
          "name": "num_blobs",
          "option": "optional"
        },
        {
          "description": "Maximal number of elements the queue can hold at any given point",
          "name": "capacity",
          "option": "optional"
        }
      ],
      "description": "\nCreates the Queue.\n",
      "outputs": [
        {
          "description": "object representing the queue",
          "name": "queue"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GE",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise greater or equal than comparison **>=** (with limited broadcast support).\n\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"GE\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", np.array([1, 5, 2, 9, 12, 3]))\nworkspace.FeedBlob(\"B\", np.array([1, 3, 4, 9, 12, 8]))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\n\nA: [ 1  5  2  9 12  3]\nB: [ 1  3  4  9 12  8]\nC: [ True  True False  True  True False]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<bool>`)* First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<bool>`)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<bool>`)* Output tensor with same dimensions as `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SinusoidPositionEncoding",
    "schema": {
      "attributes": [
        {
          "description": "Desired embedding size/number of dimensions -- defaults to 100",
          "name": "embedding_size",
          "option": "optional"
        },
        {
          "description": "Sinusoid tuning parameter -- defaults to 10000",
          "name": "alpha",
          "option": "optional"
        },
        {
          "description": "Amplitude of Sin/Cos output",
          "name": "amplitude",
          "option": "optional"
        }
      ],
      "description": "\nCalculates a sinusoid position encoding tensor as described\nin https://arxiv.org/abs/1706.03762. Takes a 2-D tensor\n(of size M x K) of positions as input, the embedding size\nas an argument, and outputs a position encoding tensor of\nsize (M x K x embedding_size). Here M is typically the max\nsequence length and K is typically the batch size.\nThe input tensor must satisfy input[m, 0] == input[m, k] for all k.\n\nEncoded as amplitude * SIN(pos/alpha^(i/embedding_size)) if i is even,\nelse amplitude * COS(pos/alpha^(i/embedding_size)). Here, pos is the position,\nalpha and amplitude are tuning parameters, i is the current dimension for\nthe embedding, and embedding_size is the number of total dimensions in\nthe embedding.\n",
      "inputs": [
        {
          "description": "2-D tensor of positions to be encoded",
          "name": "positions"
        }
      ],
      "outputs": [
        {
          "description": "3-D tensor representing the positional encoding",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentRangeMaxGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "RoIAlignGradient",
    "schema": {
      "description": null,
      "inputs": [
        {
          "description": "See RoIPoolF.",
          "name": "X"
        },
        {
          "description": "See RoIPoolF.",
          "name": "RoIs"
        },
        {
          "description": "Gradient of forward output 0 (Y)",
          "name": "dY"
        }
      ],
      "outputs": [
        {
          "description": "Gradient of forward input 0 (X)",
          "name": "dX"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "UnsortedSegmentWeightedSum",
    "schema": {
      "attributes": [
        {
          "description": "Optional int argument specifying the number of output segments and thus the first dimension of the output",
          "name": "num_segments",
          "option": "optional"
        },
        {
          "description": "Produce also gradient for `weights`. For now it's only supported in `Lengths`-based operators",
          "name": "grad_on_weights",
          "option": "optional"
        }
      ],
      "description": "\nApplies 'WeightedSum' to each segment of input tensor. Segments ids can appear in\narbitrary order (unlike in SortedSegmentWeightedSum).\n\nSEGMENT_IDS is a vector that maps each of the first dimension slices of the\nDATA to a particular group (segment). Values belonging to the same segment are\naggregated together.\n\nIf `num_segments` argument is passed it would be used as a first dimension for\nthe output. Otherwise, it'd be dynamically calculated from as the max value of\nSEGMENT_IDS plus one. Other output dimensions are inherited from the input\ntensor.\n\nInput slices are first scaled by SCALARS and then summed element-wise. It doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor for the summation",
          "name": "DATA"
        },
        {
          "description": "Scalar multipliers for the input slices. Must be a vector with the length matching the number of slices",
          "name": "SCALARS"
        },
        {
          "description": "Integer vector with the same length as the first dimension of DATA that maps each slice of DATA to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of equal to the number of segments.",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BatchMatMul",
    "schema": {
      "attributes": [
        {
          "description": "Pass 1 to transpose the last two dimensions of A before doing multiplication",
          "name": "trans_a",
          "option": "optional"
        },
        {
          "description": "Pass 1 to transpose the last two dimensions of B before doing multiplication",
          "name": "trans_b",
          "option": "optional"
        },
        {
          "description": "Pass 1 to allow broadcasting of dimensions. Behavior is the same as numpy.matmul. Gradient is currently not supported when running in broadcast mode.",
          "name": "broadcast",
          "option": "optional"
        }
      ],
      "description": "\nBatch Matrix multiplication Yi = Ai * Bi, where A has shape (dim0, dim1, ... M, K),\nB has shape (dim0, dim1, ... K, N), Y has shape (dim0, dim1, ... M, N) and i ranges\nfrom 0 to (dim0 * dim1 ...) - 1. rank(A) == rank(B) >= 2. In case of A and B being\ntwo dimensional, it behaves like normal matrix multiplication.\n",
      "inputs": [
        {
          "description": "tensor of shape (dim0, dim1 ... M, K)",
          "name": "A"
        },
        {
          "description": "tensor of shape (dim0, dim1 ... K, N)",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "tensor of shape (dim0, dim1 ... M, N)",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LpNormGradient",
    "schema": {
      "attributes": [
        {
          "description": "Order of the norm in p-norm",
          "name": "p",
          "option": "optional"
        },
        {
          "description": "whehther we calculate norm or averaged_norm.The Lp_averaged_norm(x) is defined asLp_averaged_normgradient(x) = LpNormGradient(x) / size(x)",
          "name": "average",
          "option": "optional"
        }
      ],
      "description": "\nGiven one input float tensor X, derivative dout, and produces one output\nfloat tensor dX. dX is the derivative of the Lp norm of tensor X, computed as\ndx = d(sum over |x^p|)/dx, in which p is either 1 or 2(currently only\nsupports l1 and l2 norm) determined by the argument p.\n",
      "inputs": [
        {
          "description": "1D input tensor",
          "name": "X"
        },
        {
          "description": "1D input tensor",
          "name": "dout"
        }
      ],
      "outputs": [
        {
          "description": "1D output tensor",
          "name": "dx"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "DotProduct",
    "schema": {
      "description": "\nComputes and outputs the dot product of the two input float tensors `X` and `Y`.\nNote that `X` and `Y` must be either 1D or 2D, and they must be the same shape.\nThe output tensor is 1D, which represents either the product of each element in\na respective dimension if the inputs are 1D, or the sum of the products in a\ngiven dimension if the inputs are 2D matrices. Note that the actual dot product\nis a scalar value, which is effectively the sum of the elements in the 1D\noutput tensor.\n\nFor 1D inputs:\nGiven two vectors $X = [x_0, x_1, x_2]$ and $Y = [y_0, y_1, y_2]$; $Z = [x_0 * y_0, x_1 * y_1, x_2 * y_2]$\n\nFor 2D inputs:\nGiven two matrices:\n$$X = [[x_0^0, x_1^0, x_2^0], \\\\ [x_0^1, x_1^1, x_2^1], \\\\ [x_0^2, x_1^2, x_2^2], \\\\ ..., \\\\ [x_0^n, x_1^n, x_2^n]]$$\n\nand\n\n$$Y = [[y_0^0, y_1^0, y_2^0], \\\\ [y_0^1, y_1^1, y_2^1], \\\\ [y_0^2, y_1^2, y_2^2], \\\\ ..., \\\\ [y_0^n, y_1^n, y_2^n]]$$\n\nthen\n\n$$Z =  \\biggl[\\Big((x_0^0 * y_0^0) + (x_1^0 * y_1^0) + (x_2^0 * y_2^0)\\Big), \\\\ \\Big((x_0^1 * y_0^1) + (x_1^1 * y_1^1) + (x_2^1 * y_2^1)\\Big), \\\\ \\Big((x_0^2 * y_0^2) + (x_1^2 * y_1^2) + (x_2^2 * y_2^2)\\Big), \\\\ ..., \\\\ \\Big((x_0^n * y_0^n) + (x_1^n * y_1^n) + (x_2^n * y_2^n)\\Big)\\biggr]$$\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/distance_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"DotProduct\",\n    [\"X\",  \"Y\"],\n    [\"Z\"]\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(20, size=(5)).astype(np.float32))\nworkspace.FeedBlob(\"Y\", np.random.randint(20, size=(5)).astype(np.float32))\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"))\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Z:\\n\", workspace.FetchBlob(\"X\"))\n\n\nworkspace.ResetWorkspace()\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(3,3)).astype(np.float32))\nworkspace.FeedBlob(\"Y\", np.random.randint(10, size=(3,3)).astype(np.float32))\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"))\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Z:\\n\", workspace.FetchBlob(\"Z\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [ 2. 15.  2.  7. 12.]\nY:\n [ 3. 12.  9.  3. 18.]\nZ:\n [ 2. 15.  2.  7. 12.]\nX:\n [[2. 0. 4.]\n [7. 7. 4.]\n [7. 9. 9.]]\nY:\n [[2. 0. 8.]\n [9. 6. 1.]\n [7. 8. 0.]]\nZ:\n [ 36. 109. 121.]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* 1D or 2D input tensor.",
          "name": "X"
        },
        {
          "description": "*(type: Tensor`<float>`)* 1D or 2D input tensor (must have the same shape as X).",
          "name": "Y"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* 1D output tensor.",
          "name": "Z"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "NGramFromCategorical",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Int8Slice",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        },
        {
          "description": "List of starting indices",
          "name": "starts",
          "option": "optional"
        },
        {
          "description": "List of ending indices",
          "name": "ends",
          "option": "optional"
        },
        {
          "description": "(Optional) The dimension to slice over. If specified start_idx and end_idx should also be given and it takes precedence over starts and ends",
          "name": "dim",
          "option": "optional"
        },
        {
          "description": "(Optional) The dimension to start slice from. Default is 0",
          "name": "start_idx",
          "option": "optional"
        },
        {
          "description": "(Optional) The dimension to end the slice. Default is -1",
          "name": "end_idx",
          "option": "optional"
        }
      ],
      "description": "\nProduces a slice of the input Int8 tensor. Currently, only slicing in a single\ndimension is supported.\nSlices are passed as 2 1D vectors or as two keyword argument lists with starting\nand end indices for each dimension of the input `data` tensor. If a negative\nvalue is passed for any of the start or end indices, it represents the number of\nelements before the end of that dimension. End indices are non-inclusive unless\nnegative (end index -1 means up to and including the last element).\n\n\nExample:\n\n  data = [\n      [1, 2, 3, 4],\n      [5, 6, 7, 8],\n  ]\n  starts = [0, 1]\n  ends = [-1, 3]\n\n  result = [\n      [2, 3],\n      [6, 7],\n  ]\n",
      "inputs": [
        {
          "description": "Int8 Tensor of data to extract slices from.",
          "name": "data"
        },
        {
          "description": "1D tensor: start-indices for each dimension of data.",
          "name": "starts"
        },
        {
          "description": "1D tensor: end-indices for each dimension of data.",
          "name": "ends"
        }
      ],
      "outputs": [
        {
          "description": "Sliced Int8 data tensor.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseSortedSegmentSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ReduceFrontSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ViterbiPath",
    "schema": {
      "description": "\nGiven a predictions matrix and a transitions matrix, get the path with the best\nscore\n",
      "inputs": [
        {
          "description": "N*D predictions matrix",
          "name": "predictions"
        },
        {
          "description": "D*D transitions matrix",
          "name": "transitions"
        }
      ],
      "outputs": [
        {
          "description": "N*1 vector holds the best path indices",
          "name": "viterbi_path"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceMax",
    "schema": {
      "attributes": [
        {
          "description": "A list of integers, along which to reduce.",
          "name": "axes",
          "option": "optional"
        },
        {
          "description": "Keep the reduced dimension(s) or not, default True keeps the reduced dimension(s).",
          "name": "keepdims",
          "option": "optional"
        }
      ],
      "description": "\n  Computes the max of the input tensor's element along the provided axes.\n  The resulted tensor has the same rank as the input if keepdims equal True.\n  If keepdims equal false, then the resulted tensor have the reduced dimension\n  pruned.\n",
      "inputs": [
        {
          "description": "An input tensor.",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "Reduced output tensor.",
          "name": "reduced"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsWeightedMean8BitsRowwise",
    "schema": {
      "description": "\nVariation of SparseLengthsWeightedMean operator, where\nDATA is stored using 8bits. DATA was quantized with 8Bit row-wise\nquantization (see doc to FloatToRowwiseQuantized8Bits operator). To\nrestore DATA from 8Bit, we use additional input that stores scales\nand biases.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToRowwiseQuantized8Bits",
          "name": "DATA"
        },
        {
          "description": "Scalar multipliers for the input slices. Must be a vector with the length matching the length of INDICES",
          "name": "SCALARS"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Matrix of floats, each row r_i of which stores a pair s_i, b_i -- scale and bias for i-th row",
          "name": "scale_bias"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Slice",
    "schema": {
      "attributes": [
        {
          "description": "(*Tuple(int)*): list of starting indices",
          "name": "starts",
          "option": "optional"
        },
        {
          "description": "(*Tuple(int)*): list of ending indices",
          "name": "ends",
          "option": "optional"
        }
      ],
      "category": "Tensor",
      "description": "\nProduces a slice of the input tensor.\n\n- Currently, only slicing in a single dimension is supported.\n\n- Start and end indices are either passed as two 1D input tensors or using the `starts` and `ends` arguments.\n\n- If a negative value is passed for any of the start or end indices, it represents the number of elements before the end of that dimension. End indices are non-inclusive unless negative (end index -1 means up to and including the last element).\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/slice_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Slice\",\n    [\"X\"],\n    [\"Y\"],\n    starts=(0,1),\n    ends=(-1,3)\n)\n\nworkspace.FeedBlob(\"X\", np.array([[1,2,3,4],[5,6,7,8]]))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[1 2 3 4]\n [5 6 7 8]]\nY:\n[[2 3]\n [6 7]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor*): tensor to extract slices from",
          "name": "X"
        },
        {
          "description": "(*Tensor`<int>`*): 1D tensor of start-indices for each dimension of data",
          "name": "starts"
        },
        {
          "description": "(*Tensor`<int>`*): 1D tensor of end-indices for each dimension of data",
          "name": "ends"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor*): sliced output tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseUnsortedSegmentMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "InstanceNormGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "UpsampleBilinearGradient",
    "schema": {
      "attributes": [
        {
          "description": "Scale along width dimension",
          "name": "width_scale",
          "option": "optional"
        },
        {
          "description": "Scale along height dimension",
          "name": "height_scale",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SortAndShuffle",
    "schema": {
      "description": "\nCompute the sorted indices given a field index to sort by and break the sorted\nindices into chunks of shuffle_size * batch_size and shuffle each chunk,\nfinally we shuffle between batches. If sort_by_field_idx is -1 we skip sort.\n\nFor example, we have data sorted as\n1,2,3,4,5,6,7,8,9,10,11,12\n\nand batchSize = 2 and shuffleSize = 3, when we shuffle we get:\n[3,1,4,6,5,2] [12,10,11,8,9,7]\n\nAfter this we will shuffle among different batches with size 2\n[3,1],[4,6],[5,2],[12,10],[11,8],[9,7]\n\nWe may end up with something like\n[9,7],[5,2],[12,10],[4,6],[3,1],[11,8]\n\nInput(0) is a blob pointing to a TreeCursor, and\n[Input(1),... Input(num_fields)] a list of tensors containing the data for\neach field of the dataset.\n\nSortAndShuffle is thread safe.\n",
      "inputs": [
        {
          "description": "A blob containing a pointer to the cursor.",
          "name": "cursor"
        },
        {
          "description": "First dataset field",
          "name": "dataset_field_0"
        }
      ],
      "outputs": [
        {
          "description": "Tensor containing sorted indices.",
          "name": "indices"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8MaxPoolRelu",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "MaxPool \nconsumes an input blob X and applies max pooling across the\nthe blob according to kernel sizes, stride sizes, and pad lengths defined by the\nConvPoolOpBase operator. Max pooling consisting of taking the maximum value of a\nsubset of the input tensor according to the kernel size and downsampling the\ndata into the output blob Y for further processing.\n",
      "inputs": [
        {
          "description": "Input data tensor from the previous operator; dimensions depend on whether the NCHW or NHWC operators are being used. For example, in the former, the input has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. The corresponding permutation of dimensions is used in the latter case.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output data tensor from max pooling across the input tensor. Dimensions will vary based on various kernel, stride, and pad sizes. Output will go through rectified linearfunction, where y = max(0, x).",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BitwiseOr",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise bitwise operation `bitwise_or` (with limited broadcast support).\nBoth input operands should be of type `bool`.\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor)* First operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor)* Output tensor. Has same dimensions as input `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8SumRelu",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "MaxPool2DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Percentile",
    "schema": {
      "description": "\n    This operator is used to find percentile representations for raw values, given a sample\n    set of raw values, labeled with their corresponding percentiles from the same distribution.\n    In particular, this operator takes as input a tensor of floats to find the percentile values\n    for, a 2D tensor of floats, where the first column of the tensor represents sampled values,\n    and the second column represents the percentile labels, and a tensor  of integers lengths.\n\n    This lengths tensor is used because the operator works on multiple sets of raw values at the same time. For\n    example, for an input:\n    original_values=[[3, 5, 3],[5, 1, 6]], lengths = [2, 1, 1], value_to_pct = [[3, 0.2], [5, 0.5], [1, 0.3], [3. 0.6]]\n\n    Our operator expects that each column i of the input tensor is sampled from distribution i. Lengths tells\n    us that the first two elements in value_to_pct are sampled from distribution 1, the next is from distribution two,\n    and the last is from distribution 3. We expect the output of our operator to give us [[0.2, 1.0, 0.6], [0.5, 0.3, 1.0]].\n\n    To calculate the percentile of an element, we check to see if its value is already mapped to\n    a percentile in value_to_pct. If so, we return that value. If not, we linearly interpolate between\n    the two closest values in value_to_pct. If the value is larger than all values in value_to_pct, we\n    return 1. If it's smaller than all the values, we return 0.\n\n",
      "inputs": [
        {
          "description": "Input 2D tensor of floats, representing the original, raw data to calculate percentiles for.",
          "name": "original_values"
        },
        {
          "description": "Sorted 2D tensor, with 2 columns. Each element in the first column is a float representing the raw value of a sample. Its corresponding element in the next column represents the percentile it maps to.",
          "name": "value_to_pct"
        },
        {
          "description": "1D tensor, representing the length of each distribution. We expect that the sum of elements of this tensor is equal to the total length of value_to_pct.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "1D tensor of floats, with the same dimensions as the flattened input tensor. Each element of this tensor, percentile_values[i], corresponds to the percentile calculated for original_values[i].",
          "name": "percentile_values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MergeSingleListFeatureTensors",
    "schema": {
      "attributes": [
        {
          "description": "feature ids",
          "name": "feature_ids",
          "option": "optional"
        }
      ],
      "description": "Merge given single-feature tensors with list features into one multi-feature tensor.\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": ".lengths",
          "name": "in1_lengths"
        },
        {
          "description": ".values",
          "name": "in1_values"
        },
        {
          "description": ".presence",
          "name": "in1_presence"
        }
      ],
      "outputs": [
        {
          "description": ".lengths",
          "name": "out_lengths"
        },
        {
          "description": ".keys",
          "name": "out_keys"
        },
        {
          "description": ".values.lengths",
          "name": "out_values_lengths"
        },
        {
          "description": ".values.values",
          "name": "out_values_values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RowwiseMaxGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "rnn_internal_apply_link",
    "schema": {
      "description": "\nInternal RNN operator.\n",
      "support_level": "default"
    }
  },
  {
    "name": "MergeSingleScalarFeatureTensors",
    "schema": {
      "attributes": [
        {
          "description": "feature ids",
          "name": "feature_ids",
          "option": "optional"
        }
      ],
      "description": "Merge given single-feature tensors with scalar features into one multi-feature tensor.\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": "",
          "name": "in1"
        },
        {
          "description": ".presence",
          "name": "in1_presence"
        }
      ],
      "outputs": [
        {
          "description": ".lengths",
          "name": "out_lengths"
        },
        {
          "description": ".keys",
          "name": "out_keys"
        },
        {
          "description": ".values",
          "name": "out_values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BRGNCHWCToPackedInt8BGRAStylizerDeprocess",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "TrimDataset",
    "schema": {
      "attributes": [
        {
          "description": "List of strings representing the string names in the formatspecified in the doc for CreateTreeCursor.",
          "name": "fields",
          "option": "optional"
        }
      ],
      "description": "\nTrim the given dataset inplace, given the dataset blobs and the field specs.\nTrimming happens such that the dataset will contain the largest possible number\nof records that is a multiple of the 'multiple_of' argument.\n",
      "support_level": "default"
    }
  },
  {
    "name": "PiecewiseLinearTransform",
    "schema": {
      "attributes": [
        {
          "description": "1-D vector of size (prediction_dimensions x (pieces+1)) contain the upper bounds of each piece of linear function. One special case is the first bound is the lower bound of whole piecewise function and we treat it the same as the left most functions. (bounds, slopes, intercepts) can be passed through either arg or input blobs.",
          "name": "bounds",
          "option": "optional"
        },
        {
          "description": "1-D vector of size (prediction_dimensions x pieces) containing the slopes of linear function",
          "name": "slopes",
          "option": "optional"
        },
        {
          "description": "1-D vector of size (prediction_dimensions x pieces) containing the intercepts of linear function",
          "name": "intercepts",
          "option": "optional"
        },
        {
          "description": "If set true, we assume the input is a Nx1 or Nx2 tensor. If it is Nx1 tensor, it is positive predictions. If the input is Nx2 tensor, its first column is negative predictions and second column is positive and negative + positive = 1. We just need one group of piecewise linear functions for the positive predictions.",
          "name": "binary",
          "option": "optional"
        }
      ],
      "description": "\nPiecewiseLinearTransform takes inputs -- predictions, a 2-D or 1-D tensor\n(Tensor) of size (batch_size x prediction_dimensions). The piecewise\nlinear functions are stored in bounds, slopes and intercepts. The output tensor\nhas the same shape of input `predictions` and contains the predictions\ntransformed by the piecewise linear functions. Each column of predictions has\nits own piecewise linear transformation functions. Therefore the size of\npiecewise function parameters are pieces x prediction_dimensions, except for\nbinary predictions where only the positive prediction needs them. Note that in\neach piece, low bound is excluded while high bound is included. Also the\npiecewise linear function must be continuous.\n\nNotes\n- If the input is binary predictions (Nx2 or Nx1 tensor), set the binary arg\nto true so that one group of piecewise linear functions is needed (see\ndetails below).\n- The transform parameters (bounds, slopes, intercepts) can be passed either\nthrough args or through input blobs.\n- If we have multiple groups of piecewise linear functions, each group has the\nsame number of pieces.\n- If a prediction is out of the bounds, it is capped to the smallest or largest\nbound.\n",
      "inputs": [
        {
          "description": "2-D tensor (Tensor) of size (num_batches x num_classes) containing scores",
          "name": "predictions"
        },
        {
          "description": "See bounds in Arg. (bounds, slopes, intercepts) can be passed through either arg or input blobs.",
          "name": "bounds (optional)"
        },
        {
          "description": "See slopes in Arg. (bounds, slopes, intercepts) can be passed through either arg or input blobs.",
          "name": "slopes (optional)"
        },
        {
          "description": "See intercepts in Arg. (bounds, slopes, intercepts) can be passed through either arg or input blobs.",
          "name": "intercepts (optional)"
        }
      ],
      "outputs": [
        {
          "description": "2-D tensor (Tensor) of size (num_batches x num_classes) containing transformed predictions",
          "name": "transforms"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CosineSimilarity",
    "schema": {
      "description": "\nThis op takes two input float tensors of the same size, $X$ and $Y$, and produces one output float tensor , $Z$, calculated as the cosine similarity between $X$ and $Y$. Recall, the cosine similarity between two tensors $X$ and $Y$ is defined as:\n\n$$\\mathbf{Z}=CosineSimilarity(\\mathbf{X},\\mathbf{Y}) = \\frac{\\mathbf{X}\\cdot\\mathbf{Y}}{\\|\\mathbf{X}\\|\\|\\mathbf{Y}\\|} = \\frac{\\sum_n^{i=1}X_iY_i}{\\sqrt{\\sum_n^{i=1}X_i^2}\\sqrt{\\sum_n^{i=1}Y_i^2}}$$\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/distance_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/distance_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"CosineSimilarity\",\n    [\"X\", \"Y\"],\n    [\"Z\"]\n)\n\n// Create X\nX = np.random.randn(3, 3)\nprint(\"X:\\n\",X)\n\n// Create Y\nY = np.random.randn(3, 3)\nprint(\"Y:\\n\",Y)\n\n// Feed X & Y into workspace\nworkspace.FeedBlob(\"X\", X.astype(np.float32))\nworkspace.FeedBlob(\"Y\", Y.astype(np.float32))\n\n// Run op\nworkspace.RunOperatorOnce(op)\n\n// Collect Output\nprint(\"Z:\\n\", workspace.FetchBlob(\"Z\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[-0.42635564 -0.23831588 -0.25515547]\n [ 1.43914719 -1.05613228  1.01717373]\n [ 0.06883105  0.33386519 -1.46648334]]\nY:\n [[-0.90648691 -0.14241514 -1.1070837 ]\n [ 0.92152729 -0.28115511 -0.17756722]\n [-0.88394254  1.34654037 -0.80080998]]\nZ:\n [-1.7849885e-23  1.7849885e-23 -1.0842022e-07]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "1D or 2D input tensor",
          "name": "X"
        },
        {
          "description": "1D or 2D input tensor (must have the same shape as X)",
          "name": "Y"
        }
      ],
      "outputs": [
        {
          "description": "1D output tensor",
          "name": "Z"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ClipTensorByScaling",
    "schema": {
      "attributes": [
        {
          "description": "Threshold to determine whether to scale down the tensor",
          "name": "threshold",
          "option": "optional"
        }
      ],
      "description": "\n    Clips the input tensor by scaling based on the input value and the threshold.\n    The value is usually the (pre-computed) norm of the tensor. If the value is\n    larger than the threshold, scaling would be performed in this way:\n\n          tensor *= (threshold / value).\n\n    An optional input called additional_threshold can be provided which\n    will scale the original threshold before it is used. That is,\n    the final threshold will become threshold * additional_threshold.\n    This op could be used for gradient clipping.\n",
      "inputs": [
        {
          "description": "Tensor of floats to be clipped.",
          "name": "input_tensor"
        },
        {
          "description": "Value to be compared against the threshold",
          "name": "val"
        },
        {
          "description": "An optional additional threshold to scale the original threshold",
          "name": "additional_threshold"
        }
      ],
      "outputs": [
        {
          "description": "Tensor of floats, which is the same size as the input tensor, representing the clipped tensor.",
          "name": "clipped"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "InstanceNorm",
    "schema": {
      "attributes": [
        {
          "default": 1e-05,
          "description": "The epsilon value to use to avoid division by zero.",
          "name": "epsilon",
          "option": "optional",
          "type": "float32"
        },
        {
          "default": "NCHW",
          "description": "Specifies the order of the input data blob, where $N$ is batch size, $C$ is number of channels, $H$ is spatial height, and $W$ is spatial width. The only other valid option is \"NHWC\".",
          "name": "order",
          "option": "optional",
          "type": "string"
        }
      ],
      "description": "\nThe *InstanceNorm* op applies Instance Normalization over a 4D input as described in [Instance Normalization: The Missing Ingredient for Fast Stylization](https://arxiv.org/abs/1607.08022).\n\n$$output = \\frac{input-\\mu_{input}}{\\sqrt{\\sigma_{input}^2} + \\epsilon}*scale + bias$$\n\nNotice, two of the outputs are optional so there are three output cases for this op. Case 1: output; Case 2: output, saved_mean; Case 3: output, saved_mean, saved_inv_stdev.\n\nGithub Links:\n\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/instance_norm_op.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/instance_norm_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"InstanceNorm\",\n    [\"input\", \"scale\", \"bias\"],\n    [\"output\"],\n    epsilon=1e-5,\n)\n\nworkspace.FeedBlob(\"input\", np.random.randn(2, 1, 3, 3).astype(np.float32))\nprint(\"input:\\n\", workspace.FetchBlob(\"input\"), \"\\n\")\n\nworkspace.FeedBlob(\"scale\", np.array([1.5]).astype(np.float32))\nprint(\"scale: \", workspace.FetchBlob(\"scale\"))\n\nworkspace.FeedBlob(\"bias\", np.array([1.]).astype(np.float32))\nprint(\"bias: \", workspace.FetchBlob(\"bias\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"output:\\n\", workspace.FetchBlob(\"output\"))\n\n```\n\n**Result**\n\n```\n\ninput:\n [[[[ 0.97856593 -1.1832817  -0.2540021 ]\n   [-1.3315694  -0.7485018   0.3787225 ]\n   [-0.6826597  -1.4637762   0.57116514]]]\n\n\n [[[-0.44948956  0.85544354 -0.9315333 ]\n   [-0.37202677 -0.22266895 -0.27194235]\n   [ 0.4948163  -0.7296504   1.3393803 ]]]]\n\nscale:  [1.5]\nbias:  [1.]\noutput:\n [[[[ 3.5017493  -0.3791256   1.2890853 ]\n   [-0.6453266   0.40137637  2.4249308 ]\n   [ 0.5195738  -0.8826599   2.7703972 ]]]\n\n\n [[[ 0.12639964  2.856744   -0.8821926 ]\n   [ 0.28847694  0.60098207  0.49788612]\n   [ 2.1021945  -0.45978796  3.869297  ]]]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "The input 4-dimensional NCHW tensor to be operated on.",
          "name": "input"
        },
        {
          "description": "The input 1-dimensional scale tensor of size *C*.",
          "name": "scale"
        },
        {
          "description": "The input 1-dimensional bias tensor of size *C*.",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "The output 4-dimensional tensor of the same shape as input.",
          "name": "output"
        },
        {
          "description": "(Optional) Saved mean used during training to speed up gradient computation. Should not be used for testing.",
          "name": "saved_mean"
        },
        {
          "description": "(Optional) Saved inverse stdev used during training to speed up gradient computation. Should not be used for testing.",
          "name": "saved_inv_stdev"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RoIAlignRotated",
    "schema": {
      "attributes": [
        {
          "description": "(float) default 1.0; Spatial scale of the input feature map X relative to the input image. E.g., 0.0625 if X has a stride of 16 w.r.t. the input image.",
          "name": "spatial_scale",
          "option": "optional"
        },
        {
          "description": "(int) default 1; Pooled output Y's height.",
          "name": "pooled_h",
          "option": "optional"
        },
        {
          "description": "(int) default 1; Pooled output Y's width.",
          "name": "pooled_w",
          "option": "optional"
        },
        {
          "description": "(int) default -1; number of sampling points in the interpolation grid used to compute the output value of each pooled output bin. If > 0, then exactly sampling_ratio x sampling_ratio grid points are used. If <= 0, then an adaptive number of grid points are used (computed as ceil(roi_width / pooled_w), and likewise for height).",
          "name": "sampling_ratio",
          "option": "optional"
        }
      ],
      "description": "\nSimilar to RoIAlign but can handle rotated region proposals.\nBased on https://arxiv.org/abs/1703.01086.\n",
      "inputs": [
        {
          "description": "4D feature map input of shape (N, C, H, W).",
          "name": "X"
        },
        {
          "description": "2D input of shape (R, 5 or 6) specifying R RoIs representing: batch index in [0, N - 1], center_x, center_y, width, height, angle. The RoI coordinates are in the coordinate system of the input image. `angle` should be specified in degrees and represents the RoI rotated counter-clockwise. For inputs corresponding to a single image, batch index can be excluded to have just 5 columns.",
          "name": "RoIs"
        }
      ],
      "outputs": [
        {
          "description": "4D output of shape (R, C, pooled_h, pooled_w). The r-th batch element is a pooled feature map cooresponding to the r-th RoI.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StringJoin",
    "schema": {
      "attributes": [
        {
          "description": "Delimiter for join (Default: \",\").",
          "name": "delimiter",
          "option": "optional"
        },
        {
          "description": "Axis for the join (either 0 or 1)",
          "name": "axis",
          "option": "optional"
        }
      ],
      "description": "\nTakes a 1-D or a 2-D tensor as input and joins elements in each row with the\nprovided delimiter. Output is a 1-D tensor of size equal to the first dimension\nof the input. Each element in the output tensor is a string of concatenated\nelements corresponding to each row in the input tensor. For 1-D input, each\nelement is treated as a row.\n",
      "inputs": [
        {
          "description": "1-D or 2-D tensor",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "1-D tensor of strings created by joining row elements from the input tensor.",
          "name": "strings"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ConvGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "GatherFused8BitRowwise",
    "schema": {
      "description": "\nPerform the same operation as Gather, but operating on 8-bit rowwise quantized\nmatrices with fused storage (where each row stores quantized values, and then\nthe scale and offset).\nDATA needs to have rank 2 and INDICES needs to have rank 1.\n",
      "inputs": [
        {
          "description": "uint8 tensor with rank 2 obtained with operator FloatToFused8BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA forthe rows that are being gathered",
          "name": "INDICES"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Lars",
    "schema": {
      "attributes": [
        {
          "description": "rescaling offset parameter",
          "name": "offset",
          "option": "optional"
        },
        {
          "description": "minimum learning rate for clipping",
          "name": "lr_min",
          "option": "optional"
        }
      ],
      "description": "\nImplement Layer-wise Adaptive Rate Scaling (LARS) with clipping. Before adding weight\ndecay, given a parameter tensor X and its gradient dX, the local learning rate\nfor X will be\n\nlocal_lr = trust * norm(X) / ( norm(dX) + wd * norm(X) + offset * norm(X) )\n\n      = trust / ( norm(dX) / norm(X) + wd + offset ),\n\nwhere offset is a preset hyper-parameter to avoid numerical issue and trust\nindicates how much we trust the layer to change its parameters during one update.\nIn this implementation, we uses l2 norm and the computed local learning rate is\nclipped based on the upper bound lr_max and the lower bound lr_min:\n\nlocal_lr = min(local_lr, lr_max) and local_lr = max(local_lr, lr_min)\n\n",
      "inputs": [
        {
          "description": "Parameter tensor",
          "name": "X"
        },
        {
          "description": "Gradient tensor",
          "name": "dX"
        },
        {
          "description": "Weight decay",
          "name": "wd"
        },
        {
          "description": "Trust",
          "name": "trust"
        },
        {
          "description": "Upper bound of learning rate",
          "name": "lr_max"
        }
      ],
      "outputs": [
        {
          "description": "Rescaled local learning rate",
          "name": "lr_rescaled"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MergeMultiMapFeatureTensorsGradient",
    "schema": {
      "description": "Explode given multi-feature tensors with map features into many.\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": ".lengths",
          "name": "in1_lengths"
        },
        {
          "description": ".values.lengths",
          "name": "in1_values_lengths"
        },
        {
          "description": ".values.values_grad",
          "name": "out_values_values_grad"
        }
      ],
      "outputs": [
        {
          "description": ".values.values_grad",
          "name": "in1_values_values_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsIndicesInGradientMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Tan",
    "schema": {
      "description": "\nCalculates the tangent of the given input tensor, element-wise.\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The tangent of the input tensor computed element-wise",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BooleanMaskLengths",
    "schema": {
      "description": "\nGiven a tensor of int32 `lengths` tensor representing segment lengths and a `mask` (boolean) tensor, return the segment lengths of the corresponding segmented tensor after **BooleanMask** is applied.\n\nIf `lengths` tensor is $[a_1, a_2, ..., a_n]$, then length of `mask` tensor must be $a_1 + a_2 + ... + a_n$.\n\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/boolean_mask_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"BooleanMaskLengths\",\n    [\"lengths\", \"mask\"],\n    [\"masked_lengths\"]\n)\n\nworkspace.FeedBlob(\"lengths\", np.array([1,3,2], dtype=np.int32))\nworkspace.FeedBlob(\"mask\", np.array([False,True,True,False,True,True]))\nprint(\"lengths:\", workspace.FetchBlob(\"lengths\"))\nprint(\"mask:\", workspace.FetchBlob(\"mask\"))\nworkspace.RunOperatorOnce(op)\nprint(\"masked_lengths:\", workspace.FetchBlob(\"masked_lengths\"))\n\n```\n\n**Result**\n\n```\n\nlengths: [1 3 2]\nmask: [False  True  True False  True  True]\nmasked_lengths: [0 2 2]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<int>`*): input tensor containing segment lengths",
          "name": "lengths"
        },
        {
          "description": "(*Tensor`<bool>`*): A 1D bool tensor of values to keep.",
          "name": "mask"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<int>`*): 1D tensor of same type as inputs that contains the sequence",
          "name": "masked_lengths"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Reciprocal",
    "schema": {
      "description": "\nPerforms element-wise reciprocal ($\\1/x$) of input tensor $X$.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reciprocal_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Reciprocal\",\n    [\"X\"],\n    [\"Y\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.randint(10, size=(3,3))).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[8. 3. 3.]\n [4. 0. 0.]\n [1. 2. 5.]]\nY:\n[[0.125 0.3333333  0.3333333 ]\n [0.25  inf        inf       ]\n [1     0.5        0.2       ]]\n\n```\n\n</details>\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SquaredL2Distance",
    "schema": {
      "description": "\nGiven two input float tensors X, Y, and produces one output float tensor\nof the L2 difference between X and Y that is computed as ||(X - Y)^2 / 2||.\n",
      "inputs": [
        {
          "description": "1D or 2D input tensor",
          "name": "X"
        },
        {
          "description": "1D or 2D input tensor (must have the same shape as X)",
          "name": "Y"
        }
      ],
      "outputs": [
        {
          "description": "1D output tensor",
          "name": "Z"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ArgMin",
    "schema": {
      "attributes": [
        {
          "default": -1,
          "description": "The axis to get argmin.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": true,
          "description": "If True (default), the output tensor shape will match the input tensor shape except the `axis` dimension equals 1. Else, the `axis` dimension of the output tensor is removed.",
          "name": "keepdims",
          "option": "optional",
          "type": "boolean"
        }
      ],
      "description": "\nRetrieve the argmin of an axis dimension specified by the `axis`\nargument. Given an input tensor and two arguments (`axis` and\n`keepdims`), returns a tensor containing the indices of the smallest\nelement along the given axis. If the `keepdims` arg is *True* (default),\nthe shape of the output tensor matches the input tensor except the\n`axis` dimension equals 1. Else, the `axis` dimension of the output\ntensor is removed.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/arg_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ArgMin\",\n    [\"X\"],\n    [\"Indices\"],\n    axis=1\n)\n\nworkspace.FeedBlob(\"X\", (np.random.randint(10, size=(5,5))).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Indices:\", workspace.FetchBlob(\"Indices\"))\n\n```\n\n**Result**\n\n```\n\nX: [[9. 4. 6. 4. 1.]\n  [5. 9. 8. 3. 4.]\n  [6. 1. 0. 2. 9.]\n  [7. 8. 2. 4. 9.]\n  [3. 9. 4. 9. 4.]]\nIndices: [[4]\n  [3]\n  [2]\n  [2]\n  [0]]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Tensor of indices for the smallest values.",
          "name": "Indices"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RecurrentNetwork",
    "schema": {
      "description": "\nRun the input network in a recurrent fashion. This can be used to\nimplement fairly general recurrent neural networks (RNNs).\n\nThe operator proceeds as follows.\n\n- First, initialized the states from the input recurrent states\n- For each timestep T, apply the links (that map offsets from input/output\ntensors into the inputs/outputs for the `step` network)\n- Finally, alias the recurrent states to the specified output blobs.\n\nThis is a fairly special-case meta-operator, and so the implementation\nis somewhat complex. It trades of generality (and frankly usability)\nagainst performance and control (compared to e.g. TF\ndynamic_rnn, Theano scan, etc).\n\nSee the usage examples for a flavor of how to use it.\n",
      "support_level": "default"
    }
  },
  {
    "name": "Broadcast",
    "schema": {
      "attributes": [
        {
          "description": "(int, default 0) the root to run broadcast from.",
          "name": "root",
          "option": "optional"
        }
      ],
      "description": "\nDoes a broadcast operation from the root node to every other node. The tensor\non each node should have been pre-created with the same shape and data type.\n",
      "inputs": [
        {
          "description": "The common world.",
          "name": "comm_world"
        },
        {
          "description": "A tensor to be broadcasted.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "In-place as input 1.",
          "name": "X"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PythonDLPackGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SpaceToBatch",
    "schema": {
      "attributes": [
        {
          "description": "(*int*): exclusive axis that divides the first and second dimension of matrix `A` (default=0)",
          "name": "pad",
          "option": "optional"
        },
        {
          "description": "(*int*): height/width of spatial blocks to be moved (default=2)",
          "name": "block_size",
          "option": "optional"
        },
        {
          "description": "(*string*): order of dimensions of input and output blobs; only \"NCHW\" order is currently supported (default=\"NCHW\")",
          "name": "order",
          "option": "optional"
        }
      ],
      "description": "\nZero-pads and then rearranges (permutes) blocks of spatial data into batch. More specifically, this op outputs a copy of the input tensor where values from the height and width dimensions are moved to the batch dimension. After the zero-padding is according to the `pad` argument, both height and width of the input must be divisible by the `block_size`. Only \"NCHW\" order is currently supported.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/space_batch_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"SpaceToBatch\",\n    [\"X\"],\n    [\"Y\"],\n    pad=2,\n    block_size=3\n)\n\nworkspace.FeedBlob(\"X\", np.random.rand(1,3,5,5).astype(np.float32))\nprint(\"X.shape:\", workspace.FetchBlob(\"X\").shape)\nworkspace.RunOperatorOnce(op)\nprint(\"Y.shape:\", workspace.FetchBlob(\"Y\").shape)\n\n```\n\n**Result**\n\n```\n\nX.shape: (1, 3, 5, 5)\nY.shape: (9, 3, 3, 3)\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor (NCHW order)",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): output tensor (NCHW order)",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BatchToSpace",
    "schema": {
      "attributes": [
        {
          "description": "(*int*): exclusive axis that divides the first and second dimension of matrix `A` (default=0)",
          "name": "pad",
          "option": "optional"
        },
        {
          "description": "(*int*): height/width of spatial blocks to be moved (default=2)",
          "name": "block_size",
          "option": "optional"
        },
        {
          "description": "(*string*): order of dimensions of input and output blobs; only \"NCHW\" order is currently supported (default=\"NCHW\")",
          "name": "order",
          "option": "optional"
        }
      ],
      "description": "\nRearranges (permutes) data from batch into blocks of spatial data, followed by cropping. This is the reverse transformation of `SpaceToBatch`. More specifically, this op outputs a copy of the input tensor where values from the batch dimension are moved in spatial blocks to the height and width dimensions, followed by cropping along the height and width dimensions. Only \"NCHW\" order is currently supported.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/space_batch_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"BatchToSpace\",\n    [\"X\"],\n    [\"Y\"],\n    pad=3\n)\n\nworkspace.FeedBlob(\"X\", np.random.rand(10,3,32,32).astype(np.float32))\nprint(\"X.shape:\", workspace.FetchBlob(\"X\").shape)\nworkspace.RunOperatorOnce(op)\nprint(\"Y.shape:\", workspace.FetchBlob(\"Y\").shape)\n\n```\n\n**Result**\n\n```\n\nX.shape: (10, 3, 32, 32)\nY.shape: (2, 3, 58, 58)\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor (NCHW order)",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): output tensor (NCHW order)",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Erf",
    "schema": {
      "description": "\nCalculates the arcsine of the given input tensor, element-wise.\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The arcsine of the input tensor computed element-wise",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AtomicAppend",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "GivenTensorBoolFill",
    "schema": {
      "attributes": [
        {
          "description": "The value for the elements of the output tensor.",
          "name": "values"
        },
        {
          "description": "The shape of the output tensor.Cannot set the shape argument and pass in an input at the same time.",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "The additional dimensions appended at the end of the shape indicatedby the input blob.Cannot set the extra_shape argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional"
        },
        {
          "description": "1D tensor containing the desired output shape. First input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ConvRelu",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "StumpFuncIndex",
    "schema": {
      "description": "\nSplit the elements and return the indices based on the given threshold.\n",
      "inputs": [
        {
          "description": "tensor of float",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "tensor of int64 indices for elements below/equal threshold",
          "name": "Index_Low"
        },
        {
          "description": "tensor of int64 indices for elements above threshold",
          "name": "Index_High"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "TopK",
    "schema": {
      "description": "\nRetrieve the top-K elements of the last dimension. \nGiven an input tensor of shape $(a_1, a_2, ..., a_n, r)$. `k` can be passed as an integer argument or a 1D tensor containing a single integer.\nReturns up to three outputs:\n\n1. Value tensor of shape $(a_1, a_2, ..., a_n, k)$ which contains the values of the top k elements along the last dimension\n2. Index tensor of shape $(a_1, a_2, ..., a_n, k)$ which contains the indices of the top k elements (original indices from the input tensor).\n3. [OPTIONAL] Flattened index tensor of shape $(a_1 * a_2 * ... * a_n * k,)$.\n\nGiven two equivalent values, this operator uses the indices along the last dimension as a tiebreaker. That is, the element with the lower index will appear first.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/top_k.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"TopK\",\n    [\"X\"],\n    [\"Values\", \"Indices\", \"Flattened_indices\"],\n    k=2\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(3,3,3)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Values:\", workspace.FetchBlob(\"Values\"))\nprint(\"Indices:\", workspace.FetchBlob(\"Indices\"))\nprint(\"Flattened_indices:\", workspace.FetchBlob(\"Flattened_indices\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[[6. 7. 0.]\n  [8. 7. 7.]\n  [1. 5. 6.]]\n\n [[0. 6. 1.]\n  [2. 8. 4.]\n  [1. 2. 9.]]\n\n [[4. 3. 7.]\n  [0. 1. 7.]\n  [0. 1. 8.]]]\nValues:\n[[[7. 6.]\n  [8. 7.]\n  [6. 5.]]\n\n [[6. 1.]\n  [8. 4.]\n  [9. 2.]]\n\n [[7. 4.]\n  [7. 1.]\n  [8. 1.]]]\nIndices:\n[[[1 0]\n  [0 1]\n  [2 1]]\n\n [[1 2]\n  [1 2]\n  [2 1]]\n\n [[2 0]\n  [2 1]\n  [2 1]]]\nFlattened_indices: [ 1  0  3  4  8  7 10 11 13 14 17 16 20 18 23 22 26 25]\n\n```\n\n</details>\n\n  ",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor of shape $(a_1, a_2, ..., a_n, r)$",
          "name": "X"
        },
        {
          "description": "(*int*): number of top elements to retrieve",
          "name": "k"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): output tensor of shape $(a_1, a_2, ..., a_n, k)$",
          "name": "Values"
        },
        {
          "description": "(*Tensor`<int>`*): tensor of indices of shape $(a_1, a_2, ..., a_n, k)$; indices values refer to each element's index in the last dimension of the `X` input tensor",
          "name": "Indices"
        },
        {
          "description": "(*Tensor`<int>`*): tensor of indices of shape $(a_1 * a_2 * ... * a_n * k,)$; indices values refer to each element's index in the flattened input tensor `X`",
          "name": "Flattened_indices"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SpatialBNGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ThrowChildThreadException",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CheckDatasetConsistency",
    "schema": {
      "attributes": [
        {
          "description": "List of strings representing the string names in the formatspecified in the doc for CreateTreeCursor.",
          "name": "fields",
          "option": "optional"
        }
      ],
      "description": "\nChecks that the given data fields represents a consistent dataset under\nthe schema specified by the `fields` argument. Operator fails if the fields\nare not consistent. If data is consistent, each field's data can be safely\nappended to an existing dataset, keeping it consistent.\n",
      "inputs": [
        {
          "description": "Data for field 0.",
          "name": "field_0"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RoIPoolGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CreateTextFileReader",
    "schema": {
      "attributes": [
        {
          "description": "Path to the file.",
          "name": "filename",
          "option": "optional"
        },
        {
          "description": "Number of passes over the file.",
          "name": "num_passes",
          "option": "optional"
        },
        {
          "description": "List with type of each field. Type enum is found at core.DataType.",
          "name": "field_types",
          "option": "optional"
        }
      ],
      "description": "Create a text file reader. Fields are delimited by <TAB>.",
      "outputs": [
        {
          "description": "Pointer to the created TextFileReaderInstance.",
          "name": "handler"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StringSuffix",
    "schema": {
      "attributes": [
        {
          "description": "Maximum size of the suffix, in bytes.",
          "name": "length",
          "option": "optional"
        }
      ],
      "description": "\nComputes the element-wise string suffix of the string tensor.\nInput strings that are shorter than suffix length will be returned unchanged.\nNOTE: Prefix is computed on number of bytes, which may lead to wrong behavior\nand potentially invalid strings for variable-length encodings such as utf-8.\n",
      "inputs": [
        {
          "description": "Tensor of std::string.",
          "name": "strings"
        }
      ],
      "outputs": [
        {
          "description": "Tensor of std::string containing suffixes for each output.",
          "name": "suffixes"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Expand",
    "schema": {
      "description": "\n        Broadcast the input tensor to a materialized new tensor using given shape.\n        Broadcast rule is similar to \"numpy.array(input) * numpy.ones(shape)\":\n        Dimensions are right alignment;\n        Two corresponding dimensions must have the same value, or one of them\n        equals to 1.\n        In order to align with PyTorch's `expand`, `shape` is allowed to have entries\n        equal to -1, which means to preserve the size of the corresponding dimension\n        in `X` (so it's actually equivalent to equal to 1).\n",
      "inputs": [
        {
          "description": "(*Tensor`<NumericType>`*): input tensor",
          "name": "X"
        },
        {
          "description": "(*Tensor`<int>`*): expand shape",
          "name": "shape"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<NumericType>`*): expanded tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Gelu",
    "schema": {
      "attributes": [
        {
          "description": "If true, use y = 0.5x * (1 + tanh(sqrt(2/Pi) * (x + 0.044715x^3))).",
          "name": "fast_gelu",
          "option": "optional"
        }
      ],
      "description": "\nRelu takes one input data (Tensor) and produces one output data\n(Tensor) where the rectified linear function, y = xP(X <= x) where X ~ N(0, 1),\nis applied to the tensor elementwise.\n",
      "inputs": [
        {
          "description": "1D input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "1D input tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LpNorm",
    "schema": {
      "attributes": [
        {
          "default": 2,
          "description": "Order of the norm in p-norm.",
          "name": "p",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": false,
          "description": "Whether we calculate norm or averaged_norm.The Lp_averaged_norm(x) is defined as Lp_averaged_norm(x) = LpNorm(x) / size(x)",
          "name": "average",
          "option": "optional",
          "type": "boolean"
        }
      ],
      "description": "\nThis op computes the $L_p$ norm of the one dimensional input tensor $X$, and outputs a one dimensional output tensor $Y$. Here, the $L_p$ norm is calculated as\n\n$$L_p(\\mathbf{x}) = \\sum_i x_i^p$$\n\nThis op supports $p$ values of 1 or 2. If the average argument is set, the norm is calculated as Lp_averaged_norm(x) is defined as Lp_averaged_norm(x) = LpNorm(x) / size(x).\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/lpnorm_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/lpnorm_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LpNorm\",\n    [\"X\"],\n    [\"Y\"],\n    p=2\n)\nX = np.array([5., 2.])\nprint(\"X:\\n\",X)\n\n// Feed X into workspace\nworkspace.FeedBlob(\"X\", X.astype(np.float32))\n\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [5. 2.]\nY:\n [29.]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "1D Input tensor of data to be operated on.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "1D output tensor",
          "name": "Z"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsWeightedSumFused8BitRowwise",
    "schema": {
      "description": "\nPerforms the same operation as SparseLengthsWeightedSum,\nbut operating on 8-bit rowwise quantized matrices with fused storage\n(where each row stores quantized values, and then 4-byte scale and 4-byte bias).\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused8BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Vector of weights to scale rows of DATA with before reduction",
          "name": "WEIGHTS"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseSortedSegmentMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Int8Add",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "\n    Performs element-wise binary Add (with no broadcast support).\n",
      "inputs": [
        {
          "description": "First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "Second operand. It should be of the same size as A.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "Result, has same dimensions and type as A",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AtanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentRangeSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "GetCursorOffset",
    "schema": {
      "description": "Get the current offset in the cursor.",
      "inputs": [
        {
          "description": "A blob containing a pointer to the cursor.",
          "name": "cursor"
        }
      ],
      "outputs": [
        {
          "description": "Tensor containing the offsets for the cursor.",
          "name": "offsets"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8LeakyRelu",
    "schema": {
      "attributes": [
        {
          "description": "Coefficient of leakage, default value is 0.01",
          "name": "alpha",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "\nLeakyRelu takes input data (Tensor<T>) and an argument alpha, and produces one\noutput data (Tensor<T>) where the function `f(x) = alpha * x for x < 0`,\n`f(x) = x for x >= 0`, is applied to the data tensor elementwise.\n",
      "inputs": [
        {
          "description": "1D input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "1D input tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "TanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ReduceL1Gradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "MergeMultiListFeatureTensorsGradient",
    "schema": {
      "description": "Explode given multi-feature tensors with list features into many.\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": ".lengths",
          "name": "in1_lengths"
        },
        {
          "description": ".values.lengths",
          "name": "in1_values_lengths"
        },
        {
          "description": ".values.values_grad",
          "name": "out_values_values_grad"
        }
      ],
      "outputs": [
        {
          "description": ".values.values_grad",
          "name": "in1_values_values_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8GivenIntTensorFill",
    "schema": {
      "attributes": [
        {
          "description": "Input array of type int32",
          "name": "values",
          "option": "optional"
        },
        {
          "description": "Input tensor shape",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "\n    Creates quantized tensor of type int32 with scale and zero point info.\n",
      "outputs": [
        {
          "description": "An Int8TensorCPU with scale and zero point info",
          "name": "Tensor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "TensorProtosDBInput",
    "schema": {
      "attributes": [
        {
          "description": "(int, default 0) the number of samples in a batch. The default value of 0 means that the operator will attempt to insert the entire data in a single output blob.",
          "name": "batch_size",
          "option": "optional"
        }
      ],
      "description": "\nTensorProtosDBInput is a simple input operator that basically reads things\nfrom a db where each key-value pair stores an index as key, and a TensorProtos\nobject as value. These TensorProtos objects should have the same size, and they\nwill be grouped into batches of the given size. The DB Reader is provided as\ninput to the operator and it returns as many output tensors as the size of the\nTensorProtos object. Each output will simply be a tensor containing a batch of\ndata with size specified by the 'batch_size' argument containing data from the\ncorresponding index in the TensorProtos objects in the DB.\n",
      "inputs": [
        {
          "description": "A pre-initialized DB reader. Typically, this is obtained by calling CreateDB operator with a db_name and a db_type. The resulting output blob is a DB Reader tensor",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "The output tensor in which the batches of data are returned. The number of output tensors is equal to the size of (number of TensorProto's in) the TensorProtos objects stored in the DB as values. Each output tensor will be of size specified by the 'batch_size' argument of the operator",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RemovePadding",
    "schema": {
      "attributes": [
        {
          "description": "Outer-size of padding to remove around each range.",
          "name": "padding_width",
          "option": "optional",
          "type": "int64"
        },
        {
          "description": "[OPTIONAL] Specifies a different end-padding width. If this is not set, will use same as `padding_width`.",
          "name": "end_padding_width",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nRemove padding around the edges of each segment of the input data. This is the\nreverse operation of **AddPadding**, and uses the same arguments and conventions\nfor input and output data format.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/sequence_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\naddpad_op = core.CreateOperator(\n    \"AddPadding\",\n    [\"X\", \"lengths_add\"],\n    [\"Y\", \"lengths_out_add\"],\n    padding_width=1\n)\n\nrmpad_op = core.CreateOperator(\n    \"RemovePadding\",\n    [\"Y\", \"lengths_rm\"],\n    [\"Z\", \"lengths_out_rm\"],\n    padding_width=1\n)\n\nworkspace.FeedBlob(\"X\", (np.random.randint(20, size=(3,5))))\nworkspace.FeedBlob(\"lengths_add\", np.array([3]).astype(np.int32))\nworkspace.FeedBlob(\"lengths_rm\", np.array([5]).astype(np.int32))\n\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(addpad_op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\nprint(\"lengths_out_add:\", workspace.FetchBlob(\"lengths_out_add\"))\n\nworkspace.RunOperatorOnce(rmpad_op)\nprint(\"Z:\", workspace.FetchBlob(\"Z\"))\nprint(\"lengths_out_rm:\", workspace.FetchBlob(\"lengths_out_rm\"))\n```\n\n**Result**\n\n```\nX: [[17 19  1  9  1]\n [19  3  5 19  1]\n [16  0  0  0  4]]\nY: [[ 0  0  0  0  0]\n [17 19  1  9  1]\n [19  3  5 19  1]\n [16  0  0  0  4]\n [ 0  0  0  0  0]]\nlengths_out_add: [5]\nZ: [[17 19  1  9  1]\n [19  3  5 19  1]\n [16  0  0  0  4]]\nlengths_out_rm: [3]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "Input tensor ($T<N, D_1, ..., D_n>$).",
          "name": "data_in"
        },
        {
          "description": "*(type: Tensor`<int>`)* Number of elements in each range. sum(lengths) = N. If not provided, considers all data as a single segment.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor)* Padded data tensor ($T<N + 2*padding_width, D_1, ..., D_n>$).",
          "name": "data_out"
        },
        {
          "description": "*(type: Tensor`<int>`)* [OPTIONAL] Lengths for each padded range.",
          "name": "lengths_out"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AveragedLoss",
    "schema": {
      "description": "\nThe *AveragedLoss* op takes a single 1-D input tensor *input* and returns a single output float value *output*. The output represents the average of the values in *input*. This op is commonly used for averaging losses, hence the name, however it does not exclusively operate on losses.\n\nGithub Links:\n\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/loss_op.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/loss_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"AveragedLoss\",\n    [\"input\"],\n    [\"output\"],\n)\n\nworkspace.FeedBlob(\"input\", np.array([8, 10, 12]).astype(np.float32))\nprint(\"input:\\n\", workspace.FetchBlob(\"input\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"output: \\n\", workspace.FetchBlob(\"output\"))\n\n```\n\n**Result**\n\n```\n\ninput:\n [ 8. 10. 12.]\noutput:\n 10.0\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "The input data as Tensor",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The output tensor of size 1 containing the averaged value.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReluGradient",
    "schema": {
      "description": "\nReluGradient takes both Y and dY and uses this to update dX according to the\nchain rule and derivatives of the rectified linear function.\n",
      "support_level": "default"
    }
  },
  {
    "name": "TextFileReaderRead",
    "schema": {
      "attributes": [
        {
          "description": "Maximum number of rows to read.",
          "name": "batch_size",
          "option": "optional"
        }
      ],
      "description": "Read a batch of rows from the given text file reader instance. Expects the number of fields to be equal to the number of outputs. Each output is a 1D tensor containing the values for the given field for each row. When end of file is reached, returns empty tensors.",
      "inputs": [
        {
          "description": "Pointer to an existing TextFileReaderInstance.",
          "name": "handler"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsTile",
    "schema": {
      "description": "\nGiven DATA tensor of rank r >= 1, and LENGTHS tensor of rank 1, duplicate each\nentry of the outer-most dimension of DATA according to LENGTHS, and concatenate\nthem in an output tensor of rank r.\n\nExample:\n  DATA  = [\n      [1.0, 1.2],\n      [2.3, 3.4],\n      [4.5, 5.7],\n      [6.8, 7.9],\n  ]\n  LENGTHS = [0, 1, 3, 2]\n  OUTPUT = [\n      [2.3, 3.4],\n      [4.5, 5.7],\n      [4.5, 5.7],\n      [4.5, 5.7],\n      [6.8, 7.9],\n      [6.8, 7.9],\n  ]\n",
      "inputs": [
        {
          "description": "Tensor of rank r >= 1. First dimension must be equal to the size of lengths",
          "name": "DATA"
        },
        {
          "description": "Tensor of int32 lengths of rank 1",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "Tensor of rank r",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8ChannelShuffle",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "RowWiseSparseAdam",
    "schema": {
      "attributes": [
        {
          "description": "Default 0.9",
          "name": "beta1",
          "option": "optional"
        },
        {
          "description": "Default 0.999",
          "name": "beta2",
          "option": "optional"
        },
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\n    Computes a modified Adam Update for the sparse case.\n    Given inputs (param, moment1, moment2, indices, grad, lr, iter), runs the\n    Adam update on (param, moment1[indices], moment2[indices], lr, iter) and returns\n    (new_param, new_moment1, new_moment2), where moment2 is a 1D tensor\n    with length equal to the number of rows in param:\n    shape(moment2) == shape(param)[0]. Each element of  moment2 is\n    applied to an entire row of param, and the new moment2 values are\n    calculated by averaging across the row.\n\n    ",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "First moment history",
          "name": "moment_1"
        },
        {
          "description": "Second moment history",
          "name": "moment_2"
        },
        {
          "description": "Sparse indices",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        },
        {
          "description": "iteration number",
          "name": "iter"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated first moment",
          "name": "output_moment_1"
        },
        {
          "description": "Updated second moment",
          "name": "output_moment_2"
        },
        {
          "description": "Optional Effective gradient",
          "name": "output_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Negative",
    "schema": {
      "description": "\nComputes the element-wise negative of the input.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/negative_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Negative\",\n    [\"X\"],\n    [\"Y\"]\n)\n\nworkspace.FeedBlob(\"X\", (np.random.rand(3,3).astype(np.float32)))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX: [[0.83296907 0.61407167 0.32562155]\n [0.59304523 0.03111175 0.29365504]\n [0.09478621 0.5424558  0.73940724]]\nY: [[-0.83296907 -0.61407167 -0.32562155]\n [-0.59304523 -0.03111175 -0.29365504]\n [-0.09478621 -0.5424558  -0.73940724]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* 1D input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* 1D output tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StdDevPut",
    "schema": {
      "attributes": [
        {
          "description": "(*str*): name of the stat. If not present, then uses name of input blob",
          "name": "name",
          "option": "optional"
        },
        {
          "description": "(*int64_t*): number to multiply input values by (used when inputting floats, as stats can only receive integers",
          "name": "magnitude_expand",
          "option": "optional"
        },
        {
          "description": "(*boolean*): whether or not to clamp inputs to the max inputs allowed",
          "name": "bound",
          "option": "optional"
        },
        {
          "description": "(*float*): Optionally provide a default value for receiving empty tensors",
          "name": "default_value",
          "option": "optional"
        }
      ],
      "description": "\n      Consume a value and pushes it to the global stat registry as an standard deviation.\n\n      Github Links:\n      - https://github.com/pytorch/pytorch/blob/master/caffe2/operators/stats_put_ops.cc\n\n        ",
      "inputs": [
        {
          "description": "(*Tensor`<number>`*): A scalar tensor, representing any numeric value",
          "name": "value"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Perplexity",
    "schema": {
      "description": "\nPerplexity calculates how well a probability distribution predicts a sample.\nPerplexity takes a 1-D tensor containing a batch of probabilities. Each value\nin the tensor belongs to a different sample and represents the probability of\nthe model predicting the true label for that sample. The operator returns a\nsingle (float) perplexity value for the batch.\n",
      "inputs": [
        {
          "description": "The input data as Tensor. It contains a batch oftrue label or target probabilities",
          "name": "probabilities"
        }
      ],
      "outputs": [
        {
          "description": "The output- a single (float) perplexity value for the batch",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseUnsortedSegmentSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "OneHot",
    "schema": {
      "description": "\nThe *OneHot* op accepts two inputs *indices* and *index_size_tensor*, and produces a single output *one_hots*.  For each index in *indices* the op creates a one-hot row in *one_hots* of length *index_size_tensor* where all entries are zero except the entry at the index is 1. The size of *one_hots* is *len(indices)* x *index_size_tensor*.\n\nGithub Links:\n\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/one_hot_ops.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/one_hot_ops.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"OneHot\",\n    [\"indices\", \"index_size_tensor\"],\n    [\"one_hots\"],\n)\n\nworkspace.FeedBlob(\"indices\", np.array([0,1,2,3,4]).astype(np.long))\nprint(\"indices:\\n\", workspace.FetchBlob(\"indices\"))\n\nworkspace.FeedBlob(\"index_size_tensor\", np.array([5]).astype(np.long))\nprint(\"index_size_tensor:\\n\", workspace.FetchBlob(\"index_size_tensor\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"one_hots: \\n\", workspace.FetchBlob(\"one_hots\"))\n\n```\n\n**Result**\n\n```\n\nindices:\n [0 1 2 3 4]\nindex_size_tensor:\n [5]\none_hots:\n [[1. 0. 0. 0. 0.]\n [0. 1. 0. 0. 0.]\n [0. 0. 1. 0. 0.]\n [0. 0. 0. 1. 0.]\n [0. 0. 0. 0. 1.]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "The active index for each example in the batch.",
          "name": "indices"
        },
        {
          "description": "Scalar with the size of the index. Must be in CPU context",
          "name": "index_size_tensor"
        }
      ],
      "outputs": [
        {
          "description": "Matrix of size len(indices) x index_size",
          "name": "one_hots"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MergeSingleMapFeatureTensors",
    "schema": {
      "attributes": [
        {
          "description": "feature ids",
          "name": "feature_ids",
          "option": "optional"
        }
      ],
      "description": "Merge given single-feature tensors with map features into one multi-feature tensor.\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": ".lengths",
          "name": "in1_lengths"
        },
        {
          "description": ".keys",
          "name": "in1_keys"
        },
        {
          "description": ".values",
          "name": "in1_values"
        },
        {
          "description": ".presence",
          "name": "in1_presence"
        }
      ],
      "outputs": [
        {
          "description": ".lengths",
          "name": "out_lengths"
        },
        {
          "description": ".keys",
          "name": "out_keys"
        },
        {
          "description": ".values.lengths",
          "name": "out_values_lengths"
        },
        {
          "description": ".values.keys",
          "name": "out_values_keys"
        },
        {
          "description": ".values.values",
          "name": "out_values_values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Cbrt",
    "schema": {
      "description": null,
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor calculated as the cbrt of the input tensor, element-wise.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "TensorVectorSize",
    "schema": {
      "description": "Get the size of the input vector",
      "inputs": [
        {
          "description": "std::unique_ptr<std::vector<Tensor> >",
          "name": "tensor vector"
        }
      ],
      "outputs": [
        {
          "description": "int32_t size",
          "name": "size"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AbsGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "IncrementPut",
    "schema": {
      "attributes": [
        {
          "description": "(*str*): name of the stat. If not present, then uses name of input blob",
          "name": "name",
          "option": "optional"
        },
        {
          "description": "(*int64_t*): number to multiply input values by (used when inputting floats, as stats can only receive integers",
          "name": "magnitude_expand",
          "option": "optional"
        },
        {
          "description": "(*boolean*): whether or not to clamp inputs to the max inputs allowed",
          "name": "bound",
          "option": "optional"
        },
        {
          "description": "(*float*): Optionally provide a default value for receiving empty tensors",
          "name": "default_value",
          "option": "optional"
        }
      ],
      "description": "\n    Consume a value and pushes it to the global stat registry as an sum.\n\n    Github Links:\n    - https://github.com/pytorch/pytorch/blob/master/caffe2/operators/stats_put_ops.cc\n\n        ",
      "inputs": [
        {
          "description": "(*Tensor`<number>`*): A scalar tensor, representing any numeric value",
          "name": "value"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MSRAFill",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LC2DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Accumulate",
    "schema": {
      "attributes": [
        {
          "description": "(float, default 1.0) Accumulation multiplier",
          "name": "gamma",
          "option": "optional"
        }
      ],
      "description": "\nAccumulate operator accumulates the input tensor to the output tensor. If the\noutput tensor already has the right size, we add to it; otherwise, we first\ninitialize the output tensor to all zeros, and then do accumulation. Any\nfurther calls to the operator, given that no one else fiddles with the output\nin the interim, will do simple accumulations.\nAccumulation is done using Axpby operation as shown:\n  Y = 1*X + gamma*Y\nwhere X is the input tensor, Y is the output tensor and gamma is the multiplier\nargument.\n",
      "inputs": [
        {
          "description": "The input tensor that has to be accumulated to the output tensor. If the output size is not the same as input size, the output tensor is first reshaped and initialized to zero, and only then, accumulation is done.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Accumulated output tensor",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CheckAtomicBool",
    "schema": {
      "description": "Copy the value of an atomic<bool> to a bool",
      "inputs": [
        {
          "description": "Blob containing a unique_ptr<atomic<bool>>",
          "name": "atomic_bool"
        }
      ],
      "outputs": [
        {
          "description": "Copy of the value for the atomic<bool>",
          "name": "value"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StatRegistryCreate",
    "schema": {
      "description": "\nCreate a StatRegistry object that will contain a map of performance counters\nkeyed by name. A StatRegistry is used to gather and retrieve performance\ncounts throughout the caffe2 codebase.\n",
      "outputs": [
        {
          "description": "A Blob pointing to the newly created StatRegistry.",
          "name": "handle"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GT",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise greater than comparison **>** (with limited broadcast support).\n\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"GT\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", np.array([1, 5, 2, 9, 12, 3]))\nworkspace.FeedBlob(\"B\", np.array([1, 3, 4, 9, 12, 8]))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\n\nA: [ 1  5  2  9 12  3]\nB: [ 1  3  4  9 12  8]\nC: [False  True False False False False]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<bool>`)* First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<bool>`)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<bool>`)* Output tensor with same dimensions as `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SumSqrElements",
    "schema": {
      "attributes": [
        {
          "description": "whether to average or not",
          "name": "average",
          "option": "optional"
        }
      ],
      "description": "Sums the squares elements of the input tensor.",
      "inputs": [
        {
          "description": "Tensor to sum up",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Scalar sum of squares",
          "name": "sum"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "WeightedSample",
    "schema": {
      "description": "\nThe operator performs sampling based on the input sampling weights for\neach batch. All weights must be non-negative numbers.\nThe input is a 2-D tensor (Tensor) of size (batch_size x weights_dim).\nFor each batch, an index is randomly sampled from the distribution given by\nthe weights of the corresponding batch.\nThe output is a 1-D tensor (Tensor) of size (batch_size x 1) and\ncontains the index(es) of the sampled output.\n",
      "inputs": [
        {
          "description": "A 2-D Tensor of size (batch_size x weights_dim).All weights must be non-negative numbers.",
          "name": "sampling_weights"
        },
        {
          "description": "An optional 2-D Tensor of size (batch_size x weights_dim).Its values correspond to the sampling weights.",
          "name": "sampling_values"
        }
      ],
      "outputs": [
        {
          "description": "The output tensor contains index(es) sampled from distribution givenby the weight vector(s) in the input tensorThe output is a 1-D Tensor of size (batch_size x 1)",
          "name": "sampled_indexes"
        },
        {
          "description": "The output tensor contains value(s) selected by the sampled index(es)It is a 1-D Tensor of size (batch_size x 1)",
          "name": "sampled_values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "WeightedMultiSampling",
    "schema": {
      "attributes": [
        {
          "description": "number of samples to sample from the input data",
          "name": "num_samples",
          "option": "optional"
        }
      ],
      "description": "\nThe operator performs sampling based on the input sampling weights.\nAll weights are cummulative probability thus sorted. The output is\na 1-D tensor (Tensor). If two inputs are given, the second input\nis used to provide shape of the output sample tensor. Otherwise, we use\nargument `num_samples` to determine the number of samples to generate.\n",
      "inputs": [
        {
          "description": "An optional 1-D Tensor.Input cumulative sampling probability (such as [0.2, 0.5, 0.8, 1.5]). All weights must be non-negative numbers. Note that the last value of CDF is not necessary 1. If the last value is not 1, all values in sampling_cdf will be scaled by this number.",
          "name": "sampling_cdf"
        },
        {
          "description": "Tensor whose shape will be applied to output.",
          "name": "shape_tensor (optional)"
        }
      ],
      "outputs": [
        {
          "description": "The output tensor contains indices sampled from distribution givenby the weight vector in the input tensorThe output is a 1-D Tensor of size determined by argument`num_samples` or the second input tensor.",
          "name": "sampled_indexes"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SwishGradient",
    "schema": {
      "description": "\nSwishGradient takes X, Y and dY and uses this to update dX according to the\nchain rule and derivatives of the swish function.\n",
      "support_level": "default"
    }
  },
  {
    "name": "CrossEntropyGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LT",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise less than comparison **<** (with limited broadcast support).\n\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LT\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", np.array([1, 5, 2, 9, 12, 3]))\nworkspace.FeedBlob(\"B\", np.array([1, 3, 4, 9, 12, 8]))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\n\nA: [ 1  5  2  9 12  3]\nB: [ 1  3  4  9 12  8]\nC: [False False  True False False  True]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<bool>`)* First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<bool>`)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<bool>`)* Output tensor with same dimensions as `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Softsign",
    "schema": {
      "description": "\n*Softsign* takes one input data tensor $X$ and produces one output data $Y,$ where the softsign function, $y = \\frac{x}{1+ |x|}$, is applied to $X$ elementwise. This operation can be done in an in-place fashion too, by providing the same input and output blobs.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/softsign_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Softsign\",\n    [\"X\"],\n    [\"Y\"],\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(3, 3).astype(np.float32))\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\n\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[-1.3060539   0.7242748  -1.9907674 ]\n [-0.64802396 -0.03244735  0.7455406 ]\n [-0.298492   -0.5774271   2.8364444 ]]\n\nY:\n [[-0.5663588   0.420046   -0.6656376 ]\n [-0.39321268 -0.03142761  0.4271116 ]\n [-0.2298759  -0.36605626  0.739342  ]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "Input data blob to be operated on.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob with same shape as input",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FloatToHalf",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "HardSigmoid",
    "schema": {
      "attributes": [
        {
          "description": "float: the slope of the function. Defaults to 0.2",
          "name": "alpha",
          "option": "optional"
        },
        {
          "description": "float: the bias value of the function. Defaults to 0.5",
          "name": "beta",
          "option": "optional"
        }
      ],
      "description": "\nApplies hard sigmoid operation to the input data element-wise.\nThe HardSigmoid operation takes one input $X$, produces one output $Y$, and is defined as:\n\n$$Y = max(0,min(1,x * alpha + beta))$$\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/hard_sigmoid_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/hard_sigmoid_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"HardSigmoid\",\n    [\"X\"],\n    [\"Y\"],\n    alpha = 0.2,\n    beta = 0.5,\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(5).astype(np.float32))\nprint(\"input:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"sigmoid:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\ninput: [ 1.5744036   0.31632107  1.7842269   1.4450722  -2.1726978 ]\nhard_sigmoid: [ 0.81488073,  0.56326419,  0.85684538,  0.78901446,  0.06546044]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "1D input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "1D output tensor with same shape as input",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GivenTensorStringFill",
    "schema": {
      "attributes": [
        {
          "description": "The value for the elements of the output tensor.",
          "name": "values"
        },
        {
          "description": "The shape of the output tensor.Cannot set the shape argument and pass in an input at the same time.",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "The additional dimensions appended at the end of the shape indicatedby the input blob.Cannot set the extra_shape argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional"
        },
        {
          "description": "1D tensor containing the desired output shape. First input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "UniformIntFill",
    "schema": {
      "attributes": [
        {
          "description": "(*int*): minimum value, inclusive",
          "name": "min",
          "option": "optional"
        },
        {
          "description": "(*int*): maximum value, inclusive",
          "name": "max",
          "option": "optional"
        },
        {
          "description": "(*Tuple(int)*): shape of the output, do not set when `input_as_shape`=1",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "(*int*): set to 1 to use the first input as shape; `shape` input must be in CPU context",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": "\nFill the output tensor with int32 samples from uniform distribution [`min`, `max`].\n\n- The range can be defined either by arguments or input blobs. `min` and `max` are inclusive.\n    - If the range is given by input blobs, you also need to give the shape as input.\n    - When the range is given as arguments, this operator enforces min <= max. When the range is given as inputs, the constraint is not enforced.\n    - When the range is given as inputs and max < min, the first dimension of the output is set to 0. This behavior is allowed so that dynamically sampling indices into a dynamically sized tensor is possible.\n- The shape of the output can be given as argument or input.\n\nGithub Links:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/filler_op.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/filler_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop_1 = core.CreateOperator(\n    \"UniformIntFill\",\n    [],\n    [\"output\"],\n    min=5,\n    max=10,\n    shape=(3,3)\n)\n\nop_2 = core.CreateOperator(\n    \"UniformIntFill\",\n    [\"shape\", \"min\", \"max\"],\n    [\"output\"],\n    input_as_shape=1\n)\n\n// Test arg-based op\nworkspace.RunOperatorOnce(op_1)\nprint(\"output (op_1):\\n\", workspace.FetchBlob(\"output\"))\n\n// Test input-based op\nworkspace.ResetWorkspace()\nworkspace.FeedBlob(\"shape\", np.array([5,5]))\nworkspace.FeedBlob(\"min\", np.array(13, dtype=np.int32))\nworkspace.FeedBlob(\"max\", np.array(19, dtype=np.int32))\nworkspace.RunOperatorOnce(op_2)\nprint(\"output (op_2):\\n\", workspace.FetchBlob(\"output\"))\n\n```\n\n**Result**\n\n```\n\noutput (op_1):\n [[ 6 10  7]\n [ 5 10  6]\n [ 7  5 10]]\noutput (op_2):\n [[19 13 15 13 13]\n [14 17 14 15 15]\n [17 14 19 13 13]\n [17 18 16 13 18]\n [14 15 16 18 16]]\n\n```\n\n</details>\n\n    ",
      "inputs": [
        {
          "description": "(*Tensor`<int>`*): 1-D tensor of the shape of the output, must be used with `input_as_shape` argument",
          "name": "shape"
        },
        {
          "description": "(*Tensor`<int>`*): scalar tensor containing minimum value, inclusive",
          "name": "min"
        },
        {
          "description": "(*Tensor`<int>`*): scalar tensor containing maximum value, inclusive",
          "name": "max"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<int>`*): filled output tensor",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PackRecords",
    "schema": {
      "attributes": [
        {
          "description": "List of strings representing the string names in the formatspecified in the doc for CreateTreeCursor.",
          "name": "fields",
          "option": "optional"
        }
      ],
      "description": "\nGiven a dataset under a schema specified by the `fields` argument, pack all\nthe input tensors into one, where each tensor element represents a row of data\n(batch of size 1). This format allows easier use with the rest of Caffe2\noperators.\n",
      "outputs": [
        {
          "description": "One dimensional tensor having a complex type of SharedTensorVectorPtr. In order to reverse it back to the original input it has to be inserted into UnPackRecordsOp.",
          "name": "tensor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Conditional",
    "schema": {
      "description": "\nGiven a 1-D tensor of boolean values, apply conditional operator along the first\ndimension of DataT and DataF and return DataO. Note, DataT and DataF must\nhave the exact same shape and type.\n",
      "inputs": [
        {
          "description": "Boolean tensor to select DataT or DataF",
          "name": "Condition"
        },
        {
          "description": "Data to use when True",
          "name": "DataT"
        },
        {
          "description": "Data to use when False",
          "name": "DataF"
        }
      ],
      "outputs": [
        {
          "description": "Output data after applying ConditionalOp",
          "name": "DataO"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CopyFromCPUInput",
    "schema": {
      "description": "\nTake a CPU input tensor and copy it to an output in the current\nContext (GPU or CPU). This may involves cross-device MemCpy.\n",
      "inputs": [
        {
          "description": "The input CPU tensor.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "either a TensorCUDA or a TensorCPU",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MaxPoolGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CollectAndDistributeFpnRpnProposals",
    "schema": {
      "attributes": [
        {
          "description": "(int) ROI_CANONICAL_SCALE",
          "name": "roi_canonical_scale",
          "option": "optional"
        },
        {
          "description": "(int) ROI_CANONICAL_LEVEL",
          "name": "roi_canonical_level",
          "option": "optional"
        },
        {
          "description": "(int) ROI_MAX_LEVEL",
          "name": "roi_max_level",
          "option": "optional"
        },
        {
          "description": "(int) ROI_MIN_LEVEL",
          "name": "roi_min_level",
          "option": "optional"
        },
        {
          "description": "(int) RPN_MAX_LEVEL",
          "name": "rpn_max_level",
          "option": "optional"
        },
        {
          "description": "(int) RPN_MIN_LEVEL",
          "name": "rpn_min_level",
          "option": "optional"
        },
        {
          "description": "(int) RPN_POST_NMS_TOP_N",
          "name": "rpn_post_nms_topN",
          "option": "optional"
        }
      ],
      "description": "\nMerge RPN proposals generated at multiple FPN levels and then\ndistribute those proposals to their appropriate FPN levels for Faster RCNN.\nAn anchor at one FPN level may predict an RoI that will map to another level,\nhence the need to redistribute the proposals.\n\nOnly inference is supported. To train, please use the original Python\noperator in Detectron.\n\nInputs and outputs are examples only; if min/max levels change,\nthe number of inputs and outputs, as well as their level numbering,\nwill change.\n",
      "inputs": [
        {
          "description": "RPN proposals for FPN level 2, format (image_index, x1, y1, x2, y2). See rpn_rois documentation from GenerateProposals.",
          "name": "rpn_rois_fpn2"
        },
        {
          "description": "RPN proposals for FPN level 3, format (image_index, x1, y1, x2, y2). See rpn_rois documentation from GenerateProposals.",
          "name": "rpn_rois_fpn3"
        },
        {
          "description": "RPN proposals for FPN level 4, format (image_index, x1, y1, x2, y2). See rpn_rois documentation from GenerateProposals.",
          "name": "rpn_rois_fpn4"
        },
        {
          "description": "RPN proposals for FPN level 5, format (image_index, x1, y1, x2, y2). See rpn_rois documentation from GenerateProposals.",
          "name": "rpn_rois_fpn5"
        },
        {
          "description": "RPN proposals for FPN level 6, format (image_index, x1, y1, x2, y2). See rpn_rois documentation from GenerateProposals.",
          "name": "rpn_rois_fpn6"
        },
        {
          "description": "RPN objectness probabilities for FPN level 2. See rpn_roi_probs documentation from GenerateProposals.",
          "name": "rpn_roi_probs_fpn2"
        },
        {
          "description": "RPN objectness probabilities for FPN level 3. See rpn_roi_probs documentation from GenerateProposals.",
          "name": "rpn_roi_probs_fpn3"
        },
        {
          "description": "RPN objectness probabilities for FPN level 4. See rpn_roi_probs documentation from GenerateProposals.",
          "name": "rpn_roi_probs_fpn4"
        },
        {
          "description": "RPN objectness probabilities for FPN level 5. See rpn_roi_probs documentation from GenerateProposals.",
          "name": "rpn_roi_probs_fpn5"
        },
        {
          "description": "RPN objectness probabilities for FPN level 6. See rpn_roi_probs documentation from GenerateProposals.",
          "name": "rpn_roi_probs_fpn6"
        }
      ],
      "outputs": [
        {
          "description": "Top proposals limited to rpn_post_nms_topN total, format (image_index, x1, y1, x2, y2)",
          "name": "rois"
        },
        {
          "description": "RPN proposals for ROI level 2, format (image_index, x1, y1, x2, y2)",
          "name": "rois_fpn2"
        },
        {
          "description": "RPN proposals for ROI level 3, format (image_index, x1, y1, x2, y2)",
          "name": "rois_fpn3"
        },
        {
          "description": "RPN proposals for ROI level 4, format (image_index, x1, y1, x2, y2)",
          "name": "rois_fpn4"
        },
        {
          "description": "RPN proposals for ROI level 5, format (image_index, x1, y1, x2, y2)",
          "name": "rois_fpn5"
        },
        {
          "description": "Permutation on the concatenation of all rois_fpni, i=min...max, such that when applied the RPN RoIs are restored to their original order in the input blobs.",
          "name": "rois_idx_restore"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ScatterAssign",
    "schema": {
      "description": "\nUpdate slices of the tensor in-place by overriding current value.\n\nNote: The op pretty much ignores the exact shapes of the input arguments and\ncares only about sizes. It's done for performance consideration to avoid\nunnecessary reshapes. Only first dimension of X_0 is important, let's call it\nN. If M is the total size of X_0 and K is the size of INDICES then X_i is\nassumed to be of shape K x (M / N) regardless of the real shape.\n\nNote: Each update in INDICES is applied independently which means that if\nduplicated elements are present in INDICES arbitrary one will win.\n\nCurrently only works on CPU because of access to INDICES.\n",
      "inputs": [
        {
          "description": "Tensor to be updated.",
          "name": "DATA"
        },
        {
          "description": "1-D list of indices on the first dimensionof X_0 that need to be updated",
          "name": "INDICES"
        },
        {
          "description": "Update slices, with shape len(INDICES) + shape(X_0)[1:]",
          "name": "SLICES"
        }
      ],
      "outputs": [
        {
          "description": "Has to be exactly the same tensor as the input 0",
          "name": "DATA"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FusedRandRowwiseQuantizedToFloat",
    "schema": {
      "description": "\nDe-quantizes the result of the FloatToFusedRandRowwiseQuantized operator.\nRefer FloatToFusedRandRowwiseQuantized operator for details.\n",
      "inputs": [
        {
          "description": "Fused bitwidth, tail, min, max and quantized data",
          "name": "quantized_input"
        }
      ],
      "outputs": [
        {
          "description": "Float32 data",
          "name": "float_input"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MergeSingleMapFeatureTensorsGradient",
    "schema": {
      "description": "Explode given multi-feature tensors with map features into multiple single-feature tensor.\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": ".lengths",
          "name": "in1_lengths"
        },
        {
          "description": ".presence",
          "name": "in1_presence"
        },
        {
          "description": ".values.values_grad",
          "name": "out_values_values_grad"
        }
      ],
      "outputs": [
        {
          "description": ".values_grad",
          "name": "in1_values_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "TopKGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SinGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "GivenTensorIntFill",
    "schema": {
      "attributes": [
        {
          "description": "The value for the elements of the output tensor.",
          "name": "values"
        },
        {
          "description": "The shape of the output tensor.Cannot set the shape argument and pass in an input at the same time.",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "The additional dimensions appended at the end of the shape indicatedby the input blob.Cannot set the extra_shape argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional"
        },
        {
          "description": "1D tensor containing the desired output shape. First input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Int8MaxPool",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "MaxPool \nconsumes an input blob X and applies max pooling across the\nthe blob according to kernel sizes, stride sizes, and pad lengths defined by the\nConvPoolOpBase operator. Max pooling consisting of taking the maximum value of a\nsubset of the input tensor according to the kernel size and downsampling the\ndata into the output blob Y for further processing.\n",
      "inputs": [
        {
          "description": "Input data tensor from the previous operator; dimensions depend on whether the NCHW or NHWC operators are being used. For example, in the former, the input has size (N x C x H x W), where N is the batch size, C is the number of channels, and H and W are the height and the width of the data. The corresponding permutation of dimensions is used in the latter case.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output data tensor from max pooling across the input tensor. Dimensions will vary based on various kernel, stride, and pad sizes. Output will go through rectified linear",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8Dequantize",
    "schema": {
      "description": null,
      "inputs": [
        {
          "description": "Int8 Tensor qX.",
          "name": "qX"
        }
      ],
      "outputs": [
        {
          "description": "FP32 Tensor that represents mapped real value of qX.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Adagrad",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        },
        {
          "description": "Default 1. If it is in (0, 1), the gradient square sum is decayed by this factor.",
          "name": "decay",
          "option": "optional"
        }
      ],
      "description": "\n\nComputes the AdaGrad update for an input gradient and accumulated\nhistory. Concretely, given inputs (param, grad, moment, learning_rate),\ncomputes\n\n    new_moment = moment + square(grad)\n    effective_lr = learning_rate / (sqrt(new_moment) + epsilon)\n    update = learning_rate * grad / (sqrt(new_moment) + epsilon)\n    new_param = param + update\nand returns (new_param, new_moment).\n\nOptionally returns effective_lr and update as well.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Moment history",
          "name": "moment"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated moment",
          "name": "output_moment"
        },
        {
          "description": "(optional) Effective learning rate",
          "name": "output_effective_lr"
        },
        {
          "description": "(optional) Actual update that is applied.",
          "name": "output_update"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ConstantFill",
    "schema": {
      "attributes": [
        {
          "description": "value to populate output tensor with.",
          "name": "value",
          "option": "optional"
        },
        {
          "description": "The data type for the elements of the output tensor. Strictly must be one of the types from *DataType* enum in TensorProto.",
          "name": "dtype",
          "option": "optional",
          "type": "int64"
        },
        {
          "description": "Shape of the output tensor. Cannot pass an input blob and this arg at the same time.",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "Additional dimensions appended at the end of the shape indicated by the input blob. Cannot set thisargument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional"
        },
        {
          "description": "1D tensor containing the desired output shape. First input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": "\nThis operator fills the elements of the output tensor with a constant value\nspecified by the `value` argument.\n\n- The data type is specified by the `dtype` argument\n\n- Currently, the data types supported are *float*, *int32*, *int64*, and *bool*\n\n- If the `dtype` argument is not provided, the data type of `value` is used\n\n- The output tensor shape is either specified by the `shape` argument or will\nmatch the shape of the input tensor if one is provided (if an input tensor is\nprovided, a shape argument should not be set)\n\n- Optional additional dimensions can be appended at the end as specified by\n`extra_shape` argument\n\n- If `input_as_shape` is set to True, the input should be a 1D tensor\ncontaining the desired output shape (the dimensions specified in `extra_shape`\nwill also be appended)\n\nWhen specifying `dtype` argument, use the integer keys from the *DataType* enum\nin TensorProto:\n\n```\nmessage TensorProto {\n  ...\n  enum DataType {\n    UNDEFINED = 0;\n    FLOAT = 1;  // float\n    INT32 = 2;  // int\n    BYTE = 3;  // BYTE, when deserialized, is going to be restored as uint8.\n    STRING = 4;  // string\n    BOOL = 5;  // bool\n    UINT8 = 6;  // uint8_t\n    INT8 = 7;  // int8_t\n    UINT16 = 8;  // uint16_t\n    INT16 = 9;  // int16_t\n    INT64 = 10;  // int64_t\n    FLOAT16 = 12;  // at::Half\n    DOUBLE = 13;  // double\n  }\n```\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/filler_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ConstantFill\",\n    [],\n    [\"Y\"],\n    shape=(1,5,5)\n)\n\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nY: [[[0. 0. 0. 0. 0.]\n  [0. 0. 0. 0. 0.]\n  [0. 0. 0. 0. 0.]\n  [0. 0. 0. 0. 0.]\n  [0. 0. 0. 0. 0.]]]\n```\n</details>\n\n<details>\n<summary> <b>Example 2</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ConstantFill\",\n    [\"X\"],\n    [\"Y\"],\n    value=4.0,\n    dtype=1,\n    extra_shape=(1,2)\n)\n\nworkspace.FeedBlob(\"X\", (np.random.randint(100, size=(3,3))).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX: [[86. 30. 84.]\n [34. 51.  9.]\n [29. 86. 59.]]\nY: [[[[4. 4.]]\n\n  [[4. 4.]]\n\n  [[4. 4.]]]\n\n\n [[[4. 4.]]\n\n  [[4. 4.]]\n\n  [[4. 4.]]]\n\n\n [[[4. 4.]]\n\n  [[4. 4.]]\n\n  [[4. 4.]]]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor)* [OPTIONAL] Input tensor to provide shape information.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor)* Output tensor of constant values.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FloatToFused8BitRowwiseQuantized",
    "schema": {
      "description": "\nApplies 8-bit row-wise quantization by determining the range\n(maximum - minimum) and offset (minimum value) of each row in the input\nmatrix, and then scaling each element to an 8-bit number between 0 and\n255. To later de-quantize values, the scale (range / 255) and offset\n(bias) are stored alongside the data. More precisely, each row contains\nint8 elements for each quantized element, and the last 8 bytes\nof each row in the output matrix are a float storing the scale\nfollowed by another float containing the scale.\nFor N-dimensional input tensor, the first N-1 dimensions are interpreted as\nrows and the last dimension is interpreted as a column. For example, an\ninput tensor with dimension 5x2x4 is interpreted as 10 rows and 4 columns.\n)\n",
      "inputs": [
        {
          "description": "Float32 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Conv3D",
    "schema": {
      "description": "\nThe convolution operator consumes an input vector, a 3D filter blob\nand a bias blob and computes the output. \nThe Conv2D operator computes a 2D convolution operation over an input blob $(X)$, with a filter blob $(filter)$ and a bias blob $(bias)$, and outputs a single output blob $(Y)$. Although there are several options for order, the convention is that the input $(X)$ is a blob of shape $(N,C_{in},H_{in},W_{in})$ and the output $(Y)$ is a blob of shape $(N,C_{out},H_{out},W_{out})$. Here, $N$ is the batch size, $C$ is the number of channels, $H$ is the spatial height, and $W$ is the spatial width. For example, if your input data was a batch of five, 100x120pixel RGB images, $X$ would have shape $(5,3,120,100)$.\n\nThe $filter$ input blob may contain multiple filters and has shape $(M, C_{in}, K_H, K_W)$. Here, $M$ is the number of individual filters contained in the blob, $C_{in}$ is the number of channels of each filter (by convention in 2D convolution it is the same as the number of channels in the input), $K_H$ is the spatial height of the kernel, and $K_W$ is the spatial width of the kernel. The $bias$ blob is a vector of length $M$, where there is one bias for each filter in the $filter$ blob.\n\nGiven the shape of the input blob and the filter blob, we can calculate the shape of the output blob as follows. The number of items in the batch $N$ will stay the same. The number of channels in the output will equal the number of kernels in the filter blob, so $C_{out} = M.$ With stride and pad defined below, the spatial height and width of the output ($H_{out}$ and $W_{out}$) are calculated as\n\n$$H_{out} = \\left \\lfloor{\\frac{H_{in} - K_H + 2*pad}{stride}+1}\\right \\rfloor$$\n\n\n$$W_{out} = \\left \\lfloor{\\frac{W_{in} - K_W + 2*pad}{stride}+1}\\right \\rfloor$$\n\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Conv\",\n    [\"X\", \"filter\", \"bias\"],\n    [\"Y\"],\n    kernel=5,\n    pad=1,\n    stride=2\n)\n\n// Create X: (N,C,H,W)\ndata = np.random.randn(1,1,8,8).astype(np.float32)\nprint(\"Data shape: \",data.shape)\n\n// Create W: (M,C,Kh,Kw)\nfilters = np.random.randn(3,1,5,5).astype(np.float32)\nprint(\"Filter shape: \",filters.shape)\n\n// Create b: M\nbias = np.array([1.,1.,1.]).astype(np.float32)\nprint(\"Bias shape: \",bias.shape)\n\n// Put the inputs into the workspace\nworkspace.FeedBlob(\"X\", data)\nworkspace.FeedBlob(\"filter\", filters)\nworkspace.FeedBlob(\"bias\", bias)\n\n// Run the operator\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nData shape:  (1, 1, 8, 8)\nFilter shape:  (3, 1, 5, 5)\nBias shape:  (3,)\nY:\n [[[[  0.6406407    0.8620521    0.56461596]\n   [ -1.5042953   -0.79549205 -10.683343  ]\n   [ -0.5240259    3.4538248   -3.9564204 ]]\n\n  [[  0.6876496    4.8328524   -1.9525816 ]\n   [  1.2995434   -2.3895378    7.2670045 ]\n   [  3.9929862    1.8126237    5.4699917 ]]\n\n  [[  3.55949      4.7934155    0.76086235]\n   [  3.9588015   -1.3251319    4.413117  ]\n   [ -1.5296054   -1.4924102   -3.2552304 ]]]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "Input data blob, of shape $(N, C_{in}, H_{in}, W_{in})$, to be convolved with the kernels in the filter blob.",
          "name": "X"
        },
        {
          "description": "The filter blob, of shape $(M, C_{in}, K_H, K_W)$, containing the filters to be convolved with the data.",
          "name": "filter"
        },
        {
          "description": "The bias blob, of length $M$, containing the biases for the convolution, one bias per filter.",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob, of shape $(N, C_{out}, H_{out}, W_{out})$, that contains the result of the convolution.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LSTMUnit",
    "schema": {
      "attributes": [
        {
          "description": "Bias term to add in while calculating forget gate",
          "name": "forget_bias",
          "option": "optional"
        },
        {
          "description": "When false, the sequence lengths input is left out, and all following inputs are shifted left by one.",
          "name": "sequence_lengths",
          "option": "optional"
        }
      ],
      "description": "\nLSTMUnit computes the activations of a standard LSTM (without peephole\nconnections), in a sequence-length aware fashion.\n\nConcretely, given the (fused) inputs X (TxNxD), the previous cell\nstate (NxD), and the sequence lengths (N), computes the LSTM\nactivations, avoiding computation if the input is invalid (as in, the\nvalue at X{t][n] >= seqLengths[n].\n\n",
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentWeightedSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "GatherByKey",
    "schema": {
      "description": "\nInverse operation of Partition.\n\nTakes the original, full 'keys' tensor followed by sharded value tensors,\nand returns the full value tensor, combined using the same hash used in\nPartition.\n",
      "inputs": [
        {
          "description": "The first input is the full keys tensor (same as the first input of Partition).",
          "name": "keys"
        },
        {
          "description": "Subsequented inputs are sharded values tensors.",
          "name": "sharded_values"
        }
      ],
      "outputs": [
        {
          "description": "Reconstructed values tensor.",
          "name": "values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceMean",
    "schema": {
      "attributes": [
        {
          "description": "(*Tuple(int)*): list of axes to reduce",
          "name": "axes",
          "option": "optional"
        },
        {
          "description": "(*int*): set to 1 to keep the reduced dimension(s) (default=1), else set to 0 to not keep the reduced dimension(s)",
          "name": "keepdims",
          "option": "optional"
        }
      ],
      "description": "\nComputes the **mean** of the input tensor's elements along the provided `axes`. The resulting tensor has the same rank as the input if the `keepdims` argument equals 1 (default). If `keepdims` is set to 0, then the `axes` dimensions are pruned.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduce_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ReduceMean\",\n    [\"X\"],\n    [\"Y\"],\n    axes=(0,1),\n    keepdims=0\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(1,2,5,5)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[[[9. 0. 3. 6. 0.]\n   [3. 4. 5. 0. 9.]\n   [6. 9. 1. 1. 5.]\n   [6. 2. 3. 7. 7.]\n   [3. 1. 1. 0. 1.]]\n\n  [[4. 3. 9. 8. 1.]\n   [8. 2. 0. 4. 0.]\n   [8. 9. 9. 0. 2.]\n   [7. 2. 5. 8. 9.]\n   [5. 9. 1. 9. 0.]]]]\nY:\n[[6.5 1.5 6.  7.  0.5]\n [5.5 3.  2.5 2.  4.5]\n [7.  9.  5.  0.5 3.5]\n [6.5 2.  4.  7.5 8. ]\n [4.  5.  1.  4.5 0.5]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): reduced tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "IntIndexCreate",
    "schema": {
      "attributes": [
        {
          "description": "Max number of elements, including the zero entry.",
          "name": "max_elements",
          "option": "optional"
        }
      ],
      "description": "\nCreates a dictionary that maps int32 keys to consecutive integers\nfrom 1 to max_elements. Zero is reserved for unknown keys.\n",
      "outputs": [
        {
          "description": "Pointer to an Index instance.",
          "name": "handler"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Div",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise binary division (with limited broadcast support).\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Div\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", np.array([[18,8],[2,9]]))\nworkspace.FeedBlob(\"B\", np.array([[9,2],[3,2]]))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\n\nA:\n[[18  8]\n [ 2  9]]\nB:\n[[9 2]\n [3 2]]\nC:\n[[2 4]\n [0 4]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<float>`)* Second operand. With broadcasting can be of smaller size than A. If broadcasting is disabled it should be of the same size as A.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor with same dimensions and type as A.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "WeightedSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "TimerBegin",
    "schema": {
      "attributes": [
        {
          "description": "(*str*): name of the timer object; if not set use output name",
          "name": "counter_name",
          "option": "optional"
        }
      ],
      "description": "\nStart a wallclock timer, returning a scalar tensor containing a pointer to it. The timer is stopped by calling **TimerEnd**.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/stats_ops.cc\n\n    ",
      "outputs": [
        {
          "description": "(*Tensor`<ptr>`*): pointer to a timer object",
          "name": "timer"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "UnpackSegments",
    "schema": {
      "attributes": [
        {
          "description": "The pre-defined max_length for the packed segments",
          "name": "max_length",
          "option": "optional"
        }
      ],
      "description": "Map N+1 dim tensor to N dim based on length blob",
      "inputs": [
        {
          "description": "1-d int/long tensor contains the length in each of the input.",
          "name": "lengths"
        },
        {
          "description": "N+1 dim Tensor.",
          "name": "tensor"
        }
      ],
      "outputs": [
        {
          "description": "N dim Tensor",
          "name": "packed_tensor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ThresholdedRelu",
    "schema": {
      "attributes": [
        {
          "description": "(float) defaults to 1.0.",
          "name": "alpha",
          "option": "optional"
        }
      ],
      "description": "\nThresholdedRelu takes one input data (Tensor) and produces one output data\n(Tensor) where the rectified linear function, y = x for x > alpha, y = 0\notherwise, is applied to the tensor elementwise.\n",
      "inputs": [
        {
          "description": "1D input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "1D input tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseSortedSegmentSum",
    "schema": {
      "description": "\nPulls in slices of the input tensor, groups them into segments and applies\n'Sum' to each segment. Segments need to be sorted and contiguous. See also\nSparseUnsortedSegmentSum that doesn't have this requirement.\n\nThis op is basically Gather and SortedSegmentSum fused together.\n\nINDICES should contain integers in range 0..N-1 where N is the first dimension\nof DATA. INDICES represent which slices of DATA need to be pulled in.\n\nSEGMENT_IDS is a vector that maps each referenced slice of the DATA to a\nparticular group (segment). Values belonging to the same segment are aggregated\ntogether. SEGMENT_IDS should have the same dimension as INDICES.\n\nThe first dimension of the output is equal to the number of input segments,\ni.e. `SEGMENT_IDS[-1]+1`. Other dimensions are inherited from the input tensor.\n\nSummation is done element-wise across slices of the input tensor and doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same length as INDICES and values in the range 0..K-1 and in increasing order that maps each slice of DATA referenced by INDICES to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of K (the number of segments).",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Checkpoint",
    "schema": {
      "attributes": [
        {
          "description": "(int, default 0) if set, use the db path directly and do not prepend the current root folder of the workspace.",
          "name": "absolute_path",
          "option": "optional"
        },
        {
          "description": "(string) a template string that one can combine with the iteration to create the final db name. For example, \"/home/lonestarr/checkpoint_%08d.db\"",
          "name": "db",
          "option": "optional"
        },
        {
          "description": "(string) the type of the db.",
          "name": "db_type",
          "option": "optional"
        },
        {
          "description": "(int, default 1) the checkpointing is carried out when (iter mod every) is zero.",
          "name": "every",
          "option": "optional"
        }
      ],
      "description": "\nThe Checkpoint operator is similar to the Save operator, but allows one to save\nto db every few iterations, with a db name that is appended with the iteration\ncount. It takes [1, infinity) number of inputs and has no output. The first\ninput has to be a TensorCPU of type int and has size 1 (i.e. the iteration\ncounter). This is determined whether we need to do checkpointing.\n",
      "support_level": "default"
    }
  },
  {
    "name": "LengthsIndicesInGradientMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "MaxPool1D",
    "schema": {
      "description": "MaxPool1D \nconsumes an input blob and applies max pooling across the the blob according to\nkernel sizes, stride sizes, pad lengths and dilation. Max pooling consists of\ntaking the maximum value of a subset of the input tensor according to the kernel\nsize and downsampling the data into the output blob for further processing. The\n`brew` module has a wrapper for this operator for use in a `ModelHelper` object.\n\nPooling layers reduce the spatial dimensionality of the input blob. Each of the\noutput blob's dimensions will reduce according to:\n\n$$dim_{out}=\\frac{dim_{in}-kernel+2*pad}{stride}+1$$\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"MaxPool\",\n    [\"X\"],\n    [\"Y\"],\n    kernel=2,\n    stride=2,\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(1, 1, 6, 6).astype(np.float32)) // NCHW\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX:\n [[[[-2.8534958e-01 -1.7719941e+00 -8.2277227e-04  1.1088650e+00\n    -2.1476576e+00 -3.5070452e-01]\n   [-9.0058845e-01 -3.0070004e-01 -1.7907504e+00 -7.1746534e-01\n     1.2798511e+00 -3.2214901e-01]\n   [ 1.5806322e+00  1.6845188e+00 -2.6633200e-01 -3.8576153e-01\n    -9.6424848e-02 -3.9696163e-01]\n   [ 1.2572408e-01  6.3612902e-01 -3.9554062e-01 -6.9735396e-01\n    -9.1898698e-01 -1.9609968e-01]\n   [-1.1587460e+00  2.4605224e+00 -1.5497679e+00  1.3020347e-01\n    -8.1293899e-01 -7.8803545e-01]\n   [ 1.4323474e+00  1.3618395e+00  9.8975077e-02 -1.1307785e-01\n     7.2035044e-01  2.7642491e-01]]]]\n\nY:\n [[[[-0.28534958  1.108865    1.2798511 ]\n   [ 1.6845188  -0.266332   -0.09642485]\n   [ 2.4605224   0.13020347  0.72035044]]]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor of shape NCHW or NHWC.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output data tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Atan",
    "schema": {
      "description": "\nCalculates the arctangent of the given input tensor, element-wise.\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The arctangent of the input tensor computed element-wise",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PythonDLPack",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CTCBeamSearchDecoder",
    "schema": {
      "attributes": [
        {
          "description": "Maximum number of candidates to carry over to next activation step.",
          "name": "beam_width",
          "option": "optional"
        },
        {
          "description": "Probability threshold below which outputs are ignored.",
          "name": "prune_threshold",
          "option": "optional"
        }
      ],
      "description": "Prefix beam search decoder for connectionist temporal classification.",
      "inputs": [
        {
          "description": "3D float Tensor sized [max_activation_length, batch_size, alphabet_size] of network logits (before softmax application).",
          "name": "INPUTS"
        },
        {
          "description": "(optional) 1D int vector containing sequence lengths, having size [batch_size] seq_len will be set to max_time if not provided.",
          "name": "SEQ_LEN"
        }
      ],
      "outputs": [
        {
          "description": "Output_len matrix size (batch_size * num_candidates). Each index stores lengths of candidates for its corresponding batch item.",
          "name": "OUTPUT_LEN"
        },
        {
          "description": "Values vector, size (total_decoded_outputs). The flattened vector of final output sequences, in batch order.",
          "name": "VALUES"
        },
        {
          "description": "Probability vector, size (total_decoded_outputs). Each index stores final output probability of its corresponding batch item.",
          "name": "OUTPUT_PROB"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsWeightedSumWithMainInputGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "WeightedSampleDequeueBlobs",
    "schema": {
      "attributes": [
        {
          "description": "Weights for sampling from multiple queues",
          "name": "weights",
          "option": "optional"
        },
        {
          "description": "The index of the blob (among the output blob list) that will be used to store the index of the table chosen to read the current batch.",
          "name": "table_idx_blob",
          "option": "optional"
        }
      ],
      "description": "\nDequeue the blobs from multiple queues. When one of queues is closed and empty,\nthe output status will be set to true which can be used as exit criteria for\nexecution step.\nThe 1st input is the queue and the last output is the status. The rest are\ndata blobs.\n",
      "support_level": "default"
    }
  },
  {
    "name": "SigmoidCrossEntropyWithLogitsGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Int8RoIAlign",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        },
        {
          "description": "(float) default 1.0; Spatial scale of the input feature map X relative to the input image. E.g., 0.0625 if X has a stride of 16 w.r.t. the input image.",
          "name": "spatial_scale",
          "option": "optional"
        },
        {
          "description": "(int) default 1; Pooled output Y's height.",
          "name": "pooled_h",
          "option": "optional"
        },
        {
          "description": "(int) default 1; Pooled output Y's width.",
          "name": "pooled_w",
          "option": "optional"
        },
        {
          "description": "(int) default -1; number of sampling points in the interpolation grid used to compute the output value of each pooled output bin. If > 0, then exactly sampling_ratio x sampling_ratio grid points are used. If <= 0, then an adaptive number of grid points are used (computed as ceil(roi_width / pooled_w), and likewise for height).",
          "name": "sampling_ratio",
          "option": "optional"
        }
      ],
      "description": "\nRegion of Interest (RoI) align operation as used in Mask R-CNN.\n",
      "inputs": [
        {
          "description": "4D Int8 Tensor feature map input of shape (N, C, H, W).",
          "name": "X"
        },
        {
          "description": "2D input of shape (R, 4 or 5) specifying R RoIs representing: batch index in [0, N - 1], x1, y1, x2, y2. The RoI coordinates are in the coordinate system of the input image. For inputs corresponding to a single image, batch index can be excluded to have just 4 columns.",
          "name": "RoIs"
        }
      ],
      "outputs": [
        {
          "description": "4D Int8 Tensor output of shape (R, C, pooled_h, pooled_w). The r-th batch element is a pooled feature map cooresponding to the r-th RoI.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentRangeLogMeanExpGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LengthsSum",
    "schema": {
      "description": "\nApplies 'Sum' to each segment of the input tensor. Segments are defined\nby their *LENGTHS*. *LENGTHS* is a vector that maps each of the slices of\n*DATA* to a particular segment. Values belonging to the same segment are\naggregated together and considered for the 'Sum' operation.\n\nFor example *LENGTHS = [2, 1]* stands for segments *DATA[0..1]* and *DATA[2]*\n\nThe sum of elements in *LENGTHS* must equal the number of elements in the first\ndimension of *DATA*. The length of *OUTPUT* is equal to the number of input\nsegments, i.e. len(*LENGTHS*).\n\nSummation is done element-wise across slices of the input tensor and doesn't change the shape of the individual blocks.\n\n\nThe *LengthsSum* op takes two inputs *DATA* and *LENGTHS*, and produces a single output *OUTPUT*. The op finds the sum in each of the segments of *DATA*, where segments are defined by their lengths.\nFor example, if $DATA = [2,4,3,1,2,10]$ and $LENGTHS = [2,3,1]$ then $OUTPUT = [sum([2,4]), sum([3,1,2]), sum([10])] = [6,6,10]$.\n\nGithub Link:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/segment_reduction_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LengthsSum\",\n    [\"DATA\", \"LENGTHS\"],\n    [\"OUTPUT\"],\n)\n\nworkspace.FeedBlob(\"DATA\", np.array([2,4,3,1,2,10]).astype(np.float32))\nprint(\"DATA:\\n\", workspace.FetchBlob(\"DATA\"))\n\nworkspace.FeedBlob(\"LENGTHS\", np.array([2,3,1]).astype(np.int32))\nprint(\"LENGTHS:\\n\", workspace.FetchBlob(\"LENGTHS\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"OUTPUT: \\n\", workspace.FetchBlob(\"OUTPUT\"))\n\n```\n\n**Result**\n\n```\n\nDATA:\n [ 2.  4.  3.  1.  2. 10.]\nLENGTHS:\n [2 3 1]\nOUTPUT:\n [ 6.  6. 10.]\n\n```\n\n</details>\n\n\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of len(LENGTHS) ",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "DequeueBlobs",
    "schema": {
      "attributes": [
        {
          "description": "Timeout in secs, default: no timeout",
          "name": "timeout_secs",
          "option": "optional"
        }
      ],
      "description": "\n  Dequeue the blobs from queue.\n  ",
      "inputs": [
        {
          "description": "The shared pointer for the BlobsQueue",
          "name": "queue"
        }
      ],
      "outputs": [
        {
          "description": "The blob to store the dequeued data",
          "name": "blob"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Elu",
    "schema": {
      "attributes": [
        {
          "default": 1.0,
          "description": "Defines alpha parameter used in calculation.",
          "name": "alpha",
          "option": "optional",
          "type": "float32"
        }
      ],
      "description": "\n\nThis op implements the exponential linear unit (ELU) activation function as described in [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://arxiv.org/abs/1511.07289). The op takes an input tensor $X$ of arbitrary shape, computes the elementwise elu operation, and returns a vector $Y$ of the same shape as output. The alpha parameter may be passed as an argument, but defaults to 1. The elu operation is defined as\n\n$$y=f(x) =\\begin{cases}\\alpha(e^x-1) & x < 0 \\\\ x & otherwise\\end{cases}$$\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elu_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elu_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Elu\",\n    [\"X\"],\n    [\"Y\"],\n    alpha=1.1\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(3, 3).astype(np.float32))\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\n\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[ 0.35339102  1.1860217  -0.10710736]\n [-3.1173866  -0.1889988  -0.20330353]\n [ 1.8525308  -0.368949    0.506277  ]]\n\nY:\n [[ 0.35339102  1.1860217  -0.11172786]\n [-1.0513     -0.18943374 -0.20236646]\n [ 1.8525308  -0.33939326  0.506277  ]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "1D input tensor of data to be operated on.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "1D input tensor, calculated as described above.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Flatten",
    "schema": {
      "attributes": [
        {
          "default": 1,
          "description": "Indicates up to which input dimensions (exclusive) should be flattened to the outer dimension of the output.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nFlattens the input tensor into a 2D matrix. If input tensor has shape\n$(d_0, d_1, ..., d_n)$ then the output will have shape\n$\\bigl((d_0 * d_1 * ... * d_{(axis-1)}), (d_{axis} * d_{(axis+1)} * ... * d_n)\\bigr)$.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/flatten_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Flatten\",\n    [\"X\"],\n    [\"Y\"],\n    axis=1\n)\n\nworkspace.FeedBlob(\"X\", np.random.rand(1,3,2,2))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX: [[[[0.53432311 0.23734561]\n   [0.56481598 0.52152617]]\n\n  [[0.33662627 0.32472711]\n   [0.17939016 0.97175851]]\n\n  [[0.87226421 0.49045439]\n   [0.92470531 0.30935077]]]]\nY: [[0.53432311 0.23734561 0.56481598 0.52152617 0.33662627 0.32472711\n  0.17939016 0.97175851 0.87226421 0.49045439 0.92470531 0.30935077]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor)* Input Tensor of rank >= axis.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor)* A 2D tensor with the contents of the input tensor, with input dimensions up to `axis` flattened to the outer dimension of the output and the remaining input dimensions flattened into the inner dimension of the output.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MaxPool1DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentRangeLogSumExp",
    "schema": {
      "description": "\nApplies 'LogSumExp' to each segment of input tensor. In order to allow for more\nefficient implementation of 'LogSumExp', the input segments have to be contiguous\nand non-empty.\n\nSEGMENT_IDS is a vector that maps each of the first dimension slices of the\nDATA to a particular group (segment). Values belonging to the same segment are\naggregated together.\n\nThe first dimension of the output is equal to the number of input segments,\ni.e. `SEGMENT_IDS[-1]+1`. Other dimensions are inherited from the input tensor.\n\nLogSumExp computes the element-wise log of the sum of exponentials of input slices. Operation doesn't change the shape of individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor to be aggregated",
          "name": "DATA"
        },
        {
          "description": "Vector with the same length as the first dimension of DATA and values in the range 0..K-1 and in increasing order that maps each slice of DATA to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated tensor with the first dimension of K and the other dimentsions inherited from DATA",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SumElementsGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "RetrieveCount",
    "schema": {
      "description": "\nRetrieve the current value from the counter as an integer.\n\n  Github Links:\n  - https://github.com/pytorch/pytorch/blob/master/caffe2/operators/counter_ops.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\ncreatecounter_op = core.CreateOperator(\n    \"CreateCounter\",\n    [],\n    [\"counter\"],\n    init_count=5\n)\n\nretrievecount_op = core.CreateOperator(\n    \"RetrieveCount\",\n    [\"counter\"],\n    [\"count\"]\n)\n\ncheckcounterdone_op = core.CreateOperator(\n    \"CheckCounterDone\",\n    [\"counter\"],\n    [\"done\"]\n)\n\ncountup_op = core.CreateOperator(\n    \"CountUp\",\n    [\"counter\"],\n    [\"previous_count\"],\n)\n\ncountdown_op = core.CreateOperator(\n    \"CountDown\",\n    [\"counter\"],\n    [\"done\"],\n)\n\nresetcounter_op = core.CreateOperator(\n    \"ResetCounter\",\n    [\"counter\"],\n    [\"previous_count\"],\n    init_count=3\n)\n\n\n// Create counter\nworkspace.RunOperatorOnce(createcounter_op)\nprint(\"'counter' pointer:\", workspace.FetchBlob(\"counter\"))\n\n\n// Retrieve initial counter value\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"Initial 'count':\", workspace.FetchBlob(\"count\"))\n\n\n// Check if counter is done\nworkspace.RunOperatorOnce(checkcounterdone_op)\nprint(\"Initial 'done' value:\", workspace.FetchBlob(\"done\"))\n\n\n// Test CountUp operator\nprint(\"\\nTesting CountUp operator...\")\nfor i in range(5):\n    workspace.RunOperatorOnce(countup_op)\n    print(\"'previous_count' after CountUp:\", workspace.FetchBlob(\"previous_count\"))\n\nworkspace.RunOperatorOnce(retrievecount_op)\nprint(\"'count' value after CountUp test:\", workspace.FetchBlob(\"count\"))\n\n\n// Test CountDown operator\nprint(\"\\nTesting CountDown operator...\")\nfor i in range(11):\n    workspace.RunOperatorOnce(countdown_op)\n    workspace.RunOperatorOnce(retrievecount_op)\n    print(\"'count' value after CountDown: {}\\t'done' value: {}\".format(workspace.FetchBlob(\"count\"), workspace.FetchBlob(\"done\")))\n```\n\n**Result**\n\n```\n'counter' pointer: counter, a C++ native class of type std::__1::unique_ptr<caffe2::Counter<long long>, std::__1::default_delete<caffe2::Counter<long long> > >.\nInitial 'count': 5\nInitial 'done' value: False\n\nTesting CountUp operator...\n'previous_count' after CountUp: 5\n'previous_count' after CountUp: 6\n'previous_count' after CountUp: 7\n'previous_count' after CountUp: 8\n'previous_count' after CountUp: 9\n'count' value after CountUp test: 10\n\nTesting CountDown operator...\n'count' value after CountDown: 9        'done' value: False\n'count' value after CountDown: 8        'done' value: False\n'count' value after CountDown: 7        'done' value: False\n'count' value after CountDown: 6        'done' value: False\n'count' value after CountDown: 5        'done' value: False\n'count' value after CountDown: 4        'done' value: False\n'count' value after CountDown: 3        'done' value: False\n'count' value after CountDown: 2        'done' value: False\n'count' value after CountDown: 1        'done' value: False\n'count' value after CountDown: 0        'done' value: False\n'count' value after CountDown: -1        'done' value: True\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<ptr>`)* A blob pointing to an instance of a counter.",
          "name": "counter"
        }
      ],
      "outputs": [
        {
          "description": "*(type: int)* Current count value.",
          "name": "count"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CopyRowsToTensorGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "EnsureClipped",
    "schema": {
      "description": "\nGiven a tensor, apply clip after gradient is applied; when the param is sparse as\nindicated by valid indices and grad, in-place is required\n",
      "inputs": [
        {
          "description": "Parameters to be normalized",
          "name": "param"
        },
        {
          "description": "Sparse indices, only needed for sparse param",
          "name": "indices"
        },
        {
          "description": "Gradient computed, only needed for sparse param",
          "name": "grad"
        }
      ],
      "outputs": [
        {
          "description": "param ensured to be clipped within range",
          "name": "output_param"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "WeightedSigmoidCrossEntropyWithLogits",
    "schema": {
      "description": "\nGiven three matrices: logits, targets, weights, all of the same shape,\n(batch_size, num_classes), computes the weighted sigmoid cross entropy between\nlogits and targets. Specifically, at each position r,c, this computes\nweights[r, c] * crossentropy(sigmoid(logits[r, c]), targets[r, c]), and then\naverages over each row.\nReturns a tensor of shape (batch_size,) of losses for each example.\n",
      "inputs": [
        {
          "description": "matrix of logits for each example and class.",
          "name": "logits"
        },
        {
          "description": "matrix of targets, same shape as logits.",
          "name": "targets"
        },
        {
          "description": "matrix of weights, same shape as logits.",
          "name": "weights"
        }
      ],
      "outputs": [
        {
          "description": "Vector with the total xentropy for each example.",
          "name": "xentropy"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "IndexLoad",
    "schema": {
      "attributes": [
        {
          "description": "If set, skips the first entry of the tensor. This allows to load tensors that are aligned with an embedding, where the first entry corresponds to the default 0 index entry.",
          "name": "skip_first_entry",
          "option": "optional"
        }
      ],
      "description": "\nLoads the index from the given 1-D tensor. Elements in the tensor will be given\nconsecutive indexes starting at 1. Fails if tensor contains repeated elements.\n",
      "inputs": [
        {
          "description": "Pointer to an Index instance.",
          "name": "handle"
        },
        {
          "description": "1-D tensor with elements starting with index 1.",
          "name": "items"
        }
      ],
      "outputs": [
        {
          "description": "The input handle.",
          "name": "handle"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BatchDenseToSparse",
    "schema": {
      "description": "\nThis Op is a inverse of BatchSparseToDenseOp.\nBasically, given a `lengths` vector, a `indices` vector,\nand a dense matrix `dense`, output `value` vector so that, along with\n`lengths` vector and `indices` vector, forms a sparse representation\nof the dense matrix.\n\nA sparse matrix is represented by `lengths` vector, `indices` vector,\nand `values` vector. Each element in `lengths` vector (lengths[`i`]) represents\nthe number of indices in this batch (batch `i`).\nWith in each batch, `indices` should not have duplicate number.\n\nFor example, with input:\n\n  lengths = [2, 3, 1]\n  indices = [0, 1, 2, 3, 4, 5]\n  output = [[6, 7, 0, 0, 0,  0],\n            [0, 0, 8, 9, 10, 0],\n            [0, 0, 0, 0, 0, 11]]\n\nThe output is:\n\n  values = [6, 7, 8, 9, 10, 11]\n\nafter running this operator.\n",
      "inputs": [
        {
          "description": "Flatten lengths, Used to break down indices into per batch indices",
          "name": "lengths"
        },
        {
          "description": "Flatten indices, tensor of total size = \\sum lengths, containing the indices ",
          "name": "indices"
        },
        {
          "description": "dense 2-D tensor, first dim = len(lengths), last dim > Any(indices)",
          "name": "dense"
        }
      ],
      "outputs": [
        {
          "description": "Values, tensor of the same size as `indices` and same data type as dense tensor.",
          "name": "values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "UniformFill",
    "schema": {
      "attributes": [
        {
          "description": "(*float*): minimum value, inclusive",
          "name": "min",
          "option": "optional"
        },
        {
          "description": "(*float*): maximum value, inclusive",
          "name": "max",
          "option": "optional"
        },
        {
          "description": "(*Tuple(int)*): shape of the output, do not set when `input_as_shape`=1",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "(*int*): set to 1 to use the first input as shape; `shape` input must be in CPU context",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": "\nFill the output tensor with float samples from uniform distribution [`min`, `max`].\n\n- The range can be defined either by arguments or input blobs. `min` and `max` are inclusive.\n    - If the range is given by input blobs, you also need to give the shape as input.\n    - When the range is given as arguments, this operator enforces min <= max. When the range is given as inputs, the constraint is not enforced.\n    - When the range is given as inputs and max < min, the first dimension of the output is set to 0. This behavior is allowed so that dynamically sampling indices into a dynamically sized tensor is possible.\n- The shape of the output can be given as argument or input.\n\nGithub Links:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/filler_op.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/filler_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop_1 = core.CreateOperator(\n    \"UniformFill\",\n    [],\n    [\"output\"],\n    min=5.5,\n    max=10.5,\n    shape=(3,3)\n)\n\nop_2 = core.CreateOperator(\n    \"UniformFill\",\n    [\"shape\", \"min\", \"max\"],\n    [\"output\"],\n    input_as_shape=1\n)\n\n// Test arg-based op\nworkspace.RunOperatorOnce(op_1)\nprint(\"output (op_1):\\n\", workspace.FetchBlob(\"output\"))\n\n// Test input-based op\nworkspace.ResetWorkspace()\nworkspace.FeedBlob(\"shape\", np.array([5,5]))\nworkspace.FeedBlob(\"min\", np.array(13.8, dtype=np.float32))\nworkspace.FeedBlob(\"max\", np.array(19.3, dtype=np.float32))\nworkspace.RunOperatorOnce(op_2)\nprint(\"output (op_2):\\n\", workspace.FetchBlob(\"output\"))\n\n```\n\n**Result**\n\n```\n\noutput (op_1):\n [[8.894862  8.225005  6.7890406]\n [9.588293  7.1072135 7.7234955]\n [8.210596  6.0202913 9.665462 ]]\noutput (op_2):\n [[18.965155 15.603871 15.038921 17.14872  18.134571]\n [18.84237  17.845276 19.214737 16.970337 15.494069]\n [18.754795 16.724329 15.311974 16.962536 18.60965 ]\n [15.186268 15.264773 18.73341  19.077969 14.237255]\n [15.917589 15.844325 16.248466 17.006554 17.502048]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<int>`*): 1-D tensor of the shape of the output, must be used with `input_as_shape` argument",
          "name": "shape"
        },
        {
          "description": "(*Tensor`<float>`*): scalar tensor containing minimum value, inclusive",
          "name": "min"
        },
        {
          "description": "(*Tensor`<float>`*): scalar tensor containing maximum value, inclusive",
          "name": "max"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): filled output tensor",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ChannelStats",
    "schema": {
      "description": "\nGiven an input tensor in NCHW format, computes the sum of all elements per\nchannel and the sum of all elements squared per channel. These values can be\nreduced across multiple batches and used to obtain the mean and variance across\nthe full set of batches. Using the new mean and variance as input to SpatialBN\nhas the effect of changing the batch size over which SpatialBN is applied.\n",
      "inputs": [
        {
          "description": "The input 4-dimensional tensor of shape NCHW",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "The output 1-dimensional tensor of size C containing the sum of elements of X per channel.",
          "name": "sum"
        },
        {
          "description": "The output 1-dimensional tensor of size C containing the sum of elements squared per channel.",
          "name": "sumsq"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FCTransposed",
    "schema": {
      "description": "\nSame as FC, but weight matrix is supposed to be already pretransposed.\nFCTransposed stands for calling blass with no noTrans, noTrans\n",
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LC2D",
    "schema": {
      "description": "\nThe locally connected operator consumes an input vector, a 2D filter blob\nand a bias blob and computes the output. \nNote that other parameters, such as the stride and\nkernel size, or the pads' sizes in each direction are not necessary for input\nbecause they are provided by the ConvPoolOpBase operator. Various dimension\nchecks are done implicitly, and the sizes are specified in the Input docs for\nthis operator. As is expected, the filter is locally connected with a subset of\nthe image and the bias is added; this is done throughout the image data and the\noutput is computed. As a side note on the implementation layout:\nlocally_connected_op_impl.h is the templated implementation of the\nlocally_connected_op.h file, which is why they are separate files.\n",
      "inputs": [
        {
          "description": null,
          "name": null
        },
        {
          "description": "The filter blob that will be used in the locally connected op; has size (YH * YW * M x C x kH x kW) if order == NCHW else (YH * YW * M  * KH * KW * C), where YH and YW are the height and width of the output image, C is the number of channels, and kH and kW are the height and width of the kernel.",
          "name": "filter"
        },
        {
          "description": "The 1D bias blob that is added through the locally connected op; has size (YH * YW * M).",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob that contains the result of the locally connected op.The output dimensions are functions of the kernel size, stride size, and pad lengths.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RoIPool",
    "schema": {
      "attributes": [
        {
          "description": "If set, run in test mode and skip computation of argmaxes (used for gradient computation). Only one output tensor is produced. (Default: false).",
          "name": "is_test",
          "option": "optional"
        },
        {
          "description": "A StorageOrder string (Default: \"NCHW\").",
          "name": "order",
          "option": "optional"
        },
        {
          "description": "The pooled output height (Default: 1).",
          "name": "pooled_h",
          "option": "optional"
        },
        {
          "description": "The pooled output width (Default: 1).",
          "name": "pooled_w",
          "option": "optional"
        },
        {
          "description": "Multiplicative spatial scale factor to translate ROI coords from their input scale to the scale used when pooling (Default: 1.0).",
          "name": "spatial_scale",
          "option": "optional"
        }
      ],
      "description": "\nCarries out ROI Pooling for Faster-RCNN.\nDepending on the mode, there are multiple output cases:\n\n  Output case #1: Y, argmaxes (train mode)\n  Output case #2: Y           (test mode)\n",
      "inputs": [
        {
          "description": "The input 4-D tensor of data. Only NCHW order is currently supported.",
          "name": "X"
        },
        {
          "description": "RoIs (Regions of Interest) to pool over. Should be a 2-D tensor of shape (num_rois, 5) given as [[batch_id, x1, y1, x2, y2], ...].",
          "name": "rois"
        }
      ],
      "outputs": [
        {
          "description": "RoI pooled output 4-D tensor of shape (num_rois, channels, pooled_h, pooled_w).",
          "name": "Y"
        },
        {
          "description": "Argmaxes corresponding to indices in X used for gradient computation. Only output if arg \"is_test\" is false.",
          "name": "argmaxes"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ElementwiseLinear",
    "schema": {
      "attributes": [
        {
          "default": 1,
          "description": "Describes the axis of the inputs; defaults to one because the 0th axis most likely describes the batch size.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nThis op computes the elementwise linear combination of a batch of input vectors with a weight vector and bias vector. As input, the op takes an input tensor $X$ of shape $NxD$, a weight vector $w$ of length $D$, and a bias vector $b$ of length $D$. Here, $N$ represents the batch size and $D$ represents the length of the feature vectors. The output, $Y$, is a tensor of shape $NxD$ and is calculated as\n\n$$Y_{ij} = X_{ij}w_j + b_j \\ for \\ i\\in{N}, j\\in{D}$$\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_linear_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_linear_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ElementwiseLinear\",\n    [\"X\", \"w\", \"b\"],\n    [\"Y\"]\n)\n\n// Create X\nX = np.array([[1,2,3,4,5],[6,8,9,16,10]])\nprint(\"X:\\n\",X)\n\n// Create w\nw = np.array([1,1/2.,1/3.,1/4.,1/5.])\nprint(\"w:\\n\",w)\n\n// Create b\nb = np.array([1.,1.,1.,1.,1.])\nprint(\"b:\\n\",b)\n\n\n// Feed X & w & b into workspace\nworkspace.FeedBlob(\"X\", X.astype(np.float32))\nworkspace.FeedBlob(\"w\", w.astype(np.float32))\nworkspace.FeedBlob(\"b\", b.astype(np.float32))\n\n// Run op\nworkspace.RunOperatorOnce(op)\n\n// Collect Output\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[ 1  2  3  4  5]\n [ 6  8  9 16 10]]\nw:\n [1.  0.5  0.33333333 0.25  0.2]\nb:\n [1. 1. 1. 1. 1.]\nY:\n [[2. 2. 2. 2. 2.]\n [7. 5. 4. 5. 3.]]\n\n```\n\n</details>\n\n  ",
      "inputs": [
        {
          "description": "2D input tensor of size $NxD$. This input represents the input data to be operated on.",
          "name": "X"
        },
        {
          "description": "1D scaling factors, or weights, of size $D$. This input contains the weights that will be multiplied by the data.",
          "name": "w"
        },
        {
          "description": "1D biases of size $D$. This input contains the biases that will be added to the products of the weights and data.",
          "name": "b"
        }
      ],
      "outputs": [
        {
          "description": "2D output tensor of size $NxD$. Calculated as described above.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsSplit",
    "schema": {
      "attributes": [
        {
          "description": "Number of splits for each element in LENGTHS",
          "name": "n_split",
          "option": "optional"
        }
      ],
      "description": "\nGiven input vector LENGTHS, and input n_split, LengthsSplit returns\na single output vector. It \"splits\" each length into n_split values which add\nup to the original length. It will attempt to do equal splits, and if not possible,\nit orders larger values first. If the n_split is larger than the length, zero\npadding will be applied.\n\ne.g. LENGTHS = [9 4 5]\n     n_split = 3\n     Y = [3 3 3 2 1 1 2 2 1]\n\ne.g. LENGTHS = [2, 1, 2]\n     n_split = 3\n     Y = [1 1 0 1 0 0 1 1 0]\n",
      "inputs": [
        {
          "description": "Mx1 Input tensor denoting INT32 lengths",
          "name": "LENGTHS"
        },
        {
          "description": "(Optional) Number of splits for each element in LENGTHS (overrides argument)",
          "name": "n_split"
        }
      ],
      "outputs": [
        {
          "description": "(M*n_split)x1 Output vector denoting split lengths",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsWeightedSum",
    "schema": {
      "attributes": [
        {
          "description": "Produce also gradient for `weights`. For now it's only supported in `Lengths`-based operators",
          "name": "grad_on_weights",
          "option": "optional"
        }
      ],
      "description": "\nPulls in slices of the input tensor, groups them into segments and applies\n'WeightedSum' to each segment. Segments are defined by their LENGTHS.\n\nThis op is basically Gather and LengthsWeightedSum fused together.\n\nINDICES should contain integers in range 0..N-1 where N is the first dimension\nof DATA. INDICES represent which slices of DATA need to be pulled in.\n\nLENGTHS is a vector that defines slice sizes by first dimension of DATA. Values\nbelonging to the same segment are aggregated together. sum(LENGTHS) has\nto match INDICES size.\n\nThe first dimension of the output is equal to the number of input segment,\ni.e. `len(LENGTHS)`. Other dimensions are inherited from the input tensor.\n\nInput slices are first scaled by SCALARS and then summed element-wise. It doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor for the summation",
          "name": "DATA"
        },
        {
          "description": "Scalar multipliers for the input slices. Must be a vector with the length matching the number of slices",
          "name": "SCALARS"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Non negative vector with sum of elements equal to INDICES length",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of K (the number of segments).",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Sub",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise binary subtraction (with limited broadcast support).\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Sub\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", np.array([[10,12],[4,14]]))\nworkspace.FeedBlob(\"B\", np.array([[5,16],[1,19]]))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\n\nA:\n[[10 12]\n [ 4 14]]\nB:\n[[ 5 16]\n [ 1 19]]\nC:\n[[ 5 -4]\n [ 3 -5]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<float>`)* Second operand. With broadcasting can be of smaller size than A. If broadcasting is disabled it should be of the same size as A.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor with same dimensions and type as A.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Adadelta",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        },
        {
          "description": "Default 0.95, the squared gradient sum is decayed by this factor.",
          "name": "decay",
          "option": "optional"
        }
      ],
      "description": "\n\nComputes the AdaDelta update (https://arxiv.org/abs/1212.5701) for an input\ngradient and accumulated history of squared gradients. Concretely, given\ninputs (param, moment, moment_delta, grad, learning_rate), computes:\n\n    new_moment = moment * decay + square(grad) * (1 - decay)\n    new_grad = sqrt(moment_delta + epsilon) / sqrt(new_moment + epsilon) * grad\n    new_param = param + learning_rate * new_grad\n    new_moment_delta = moment_delta * decay + square(new_grad) * (1 - decay)\n\nand returns (new_param, new_moment, new_moment_delta).\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Average of squared gradients",
          "name": "moment"
        },
        {
          "description": "Average of squared parameter updates",
          "name": "moment_delta"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "Learning rate",
          "name": "lr"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated average squared gradient",
          "name": "output_moment"
        },
        {
          "description": "Updated average of squared parameter updates",
          "name": "output_moment_delta"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "If",
    "schema": {
      "attributes": [
        {
          "description": "Net executed when condition is true",
          "name": "then_net",
          "option": "optional"
        },
        {
          "description": "Net executed when condition is false (optional)",
          "name": "else_net",
          "option": "optional"
        }
      ],
      "description": "\n'If' control operator, first input is a scalar boolean blob that stores condition\nvalue. Accepts 'then_net' (required) and 'else_net' (optional) arguments for 'then' and\n'else' subnets respectively. Subnets are executed in the same workspace as 'If'.\n    ",
      "inputs": [
        {
          "description": "Scalar boolean condition",
          "name": "condition"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "IntegralImage",
    "schema": {
      "description": "\nComputes an integral image, which contains the sum of pixel values within\nan image vertically and horizontally. This integral image can then be used\nwith other detection and tracking techniques.\n",
      "inputs": [
        {
          "description": "Images tensor of the form (N, C, H, W)",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Integrated image of the form (N, C, H+1, W+1)",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsWeightedSum",
    "schema": {
      "attributes": [
        {
          "description": "Produce also gradient for `weights`. For now it's only supported in `Lengths`-based operators",
          "name": "grad_on_weights",
          "option": "optional"
        }
      ],
      "description": "\nApplies 'WeightedSum' to each segment of the input tensor. Segments are defined\nby their *LENGTHS*. *LENGTHS* is a vector that maps each of the slices of\n*DATA* to a particular segment. Values belonging to the same segment are\naggregated together and considered for the 'WeightedSum' operation.\n\nFor example *LENGTHS = [2, 1]* stands for segments *DATA[0..1]* and *DATA[2]*\n\nThe sum of elements in *LENGTHS* must equal the number of elements in the first\ndimension of *DATA*. The length of *OUTPUT* is equal to the number of input\nsegments, i.e. len(*LENGTHS*).\n\nInput slices are first scaled by SCALARS and then summed element-wise. It doesn't change the shape of the individual blocks.\n\n\nThe *LengthsWeightedSum* op takes three inputs *DATA*, *LENGTHS*, and *SCALARS*, and produces a single output *OUTPUT*. The op finds the weighted sum in each of the segments of *DATA*, where segments are defined by their lengths. Before calculating the sums, the input *DATA* is weighted by the contents of *SCALARS*.\nFor example, if $DATA = [2,4,3,1,2,10]$, $SCALARS = [8, 2, 1, 4, 1, 0.6]$, and $LENGTHS = [2,3,1]$, then $OUTPUT = [sum([8*2,2*4]), sum([1*3,4*1,1*2]), sum([0.6*10])] = [24,9,6]$.\n\nGithub Link:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/segment_reduction_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LengthsWeightedSum\",\n    [\"DATA\", \"SCALARS\",\"LENGTHS\"],\n    [\"OUTPUT\"],\n)\n\nworkspace.FeedBlob(\"DATA\", np.array([2,4,3,1,2,10]).astype(np.float32))\nprint(\"DATA:\\n\", workspace.FetchBlob(\"DATA\"))\n\nworkspace.FeedBlob(\"SCALARS\", np.array([8, 2, 1, 4, 1, 0.6]).astype(np.float32))\nprint(\"SCALARS:\\n\", workspace.FetchBlob(\"SCALARS\"))\n\nworkspace.FeedBlob(\"LENGTHS\", np.array([2,3,1]).astype(np.int32))\nprint(\"LENGTHS:\\n\", workspace.FetchBlob(\"LENGTHS\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"OUTPUT: \\n\", workspace.FetchBlob(\"OUTPUT\"))\n\n```\n\n**Result**\n\n```\n\nDATA:\n [ 2.  4.  3.  1.  2. 10.]\nSCALARS:\n [8.  2.  1.  4.  1.  0.6]\nLENGTHS:\n [2 3 1]\nOUTPUT:\n [24.  9.  6.]\n\n```\n\n</details>\n\n\n  ",
      "inputs": [
        {
          "description": "Input tensor for the summation",
          "name": "DATA"
        },
        {
          "description": "Scalar multipliers for the input slices. Must be a vector with the length matching the number of slices",
          "name": "SCALARS"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of len(LENGTHS) ",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Print",
    "schema": {
      "attributes": [
        {
          "description": "(bool) if 1, saves contents to the root folder of the current workspace, appending the tensor contents to a file named after the blob name. Otherwise, logs to stderr.",
          "name": "to_file",
          "option": "optional"
        },
        {
          "description": "(int, default 0) If set, prints the first `limit` elements of tensor. If 0, prints the first `k_limit_default`(1000) elements of tensor",
          "name": "limit",
          "option": "optional"
        },
        {
          "description": "(int, default 1) Print tensor every `every_n` runs",
          "name": "every_n",
          "option": "optional"
        }
      ],
      "description": "Logs shape and contents of input tensor to stderr or to a file.",
      "inputs": [
        {
          "description": "The tensor to print.",
          "name": "tensor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceFrontMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "AsinGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseToDenseMask",
    "schema": {
      "attributes": [
        {
          "description": "list(int) argument with desired ids on the 'dense' output dimension",
          "name": "mask",
          "option": "optional"
        },
        {
          "description": "bool whether to return presence mask, false by default",
          "name": "return_presence_mask",
          "option": "optional"
        }
      ],
      "description": "\nConvert sparse representations to dense with given indices.\n\nTransforms a sparse representation of map<id, value> represented as `indices`\nvector and `values` tensor into a compacted tensor where the first dimension\ncorresponds to each id provided in mask argument. Missing values are filled with\nthe value of `default_value`. After running this op:\n\n  output[j, :] = values[i] // where mask[j] == indices[i]\n  output[j, ...] = default_value // when mask[j] doesn't appear in indices\n\nIf `lengths` is provided and not empty, and extra \"batch\" dimension is prepended\nto the output.\n\n`values` and `default_value` can have additional matching dimensions, operation\nis performed on the entire subtensor in thise case.\n\nFor example, if `lengths` is supplied and `values` is 1-D vector of floats and\n`default_value` is a float scalar, the output is going to be a float matrix\nof size `len(lengths) X len(mask)`\n",
      "inputs": [
        {
          "description": "1-D int32/int64 tensor of concatenated ids of data",
          "name": "indices"
        },
        {
          "description": "Data tensor, first dimension has to match `indices`",
          "name": "values"
        },
        {
          "description": "Default value for the output if the id is not present in `indices`. Must have the same type as `values` and the same shape, but without the first dimension",
          "name": "default_value"
        },
        {
          "description": "Optional lengths to represent a batch of `indices` and `values`.",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor of the same type as `values` of shape `[len(lengths), len(mask)] + shape(default_value)` (if `lengths` is not provided the first dimension is omitted)",
          "name": "output"
        },
        {
          "description": "Bool tensor of shape `[len(lengths), len(mask)]` (if `lengths` is not provided the first dimension is omitted). True when a value for given id was present, false otherwise.",
          "name": "presence_mask"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseToDenseMaskGradient",
    "schema": {
      "description": "\nThe output is the gradient of the input value from SparseToDenseMask. The\ngradient for default_value has not been implemented.\n",
      "support_level": "default"
    }
  },
  {
    "name": "LC3D",
    "schema": {
      "description": "\nThe locally connected operator consumes an input vector, a 3D filter blob\nand a bias blob and computes the output. \nNote that other parameters, such as the stride and\nkernel size, or the pads' sizes in each direction are not necessary for input\nbecause they are provided by the ConvPoolOpBase operator. Various dimension\nchecks are done implicitly, and the sizes are specified in the Input docs for\nthis operator. As is expected, the filter is locally connected with a subset of\nthe image and the bias is added; this is done throughout the image data and the\noutput is computed. As a side note on the implementation layout:\nlocally_connected_op_impl.h is the templated implementation of the\nlocally_connected_op.h file, which is why they are separate files.\n",
      "inputs": [
        {
          "description": null,
          "name": null
        },
        {
          "description": "The filter blob that will be used in the locally connected op; has size (YH * YW * M x C x kH x kW) if order == NCHW else (YH * YW * M  * KH * KW * C), where YH and YW are the height and width of the output image, C is the number of channels, and kH and kW are the height and width of the kernel.",
          "name": "filter"
        },
        {
          "description": "The 1D bias blob that is added through the locally connected op; has size (YH * YW * M).",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob that contains the result of the locally connected op.The output dimensions are functions of the kernel size, stride size, and pad lengths.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "UpsampleBilinear",
    "schema": {
      "attributes": [
        {
          "description": "Scale along width dimension",
          "name": "width_scale",
          "option": "optional"
        },
        {
          "description": "Scale along height dimension",
          "name": "height_scale",
          "option": "optional"
        }
      ],
      "description": "\nResizes the spatial dimensions of the input using bilinear\ninterpolation. The `width_scale` and `height_scale` arguments\ncontrol the size of the output, which is given by:\noutput_width = floor(input_width * width_scale)\noutput_height = floor(output_height * height_scale)\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "X"
        },
        {
          "description": "1D, 2-element, Scales tensor, [height_scale, width_scale]",
          "name": "scales"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Append",
    "schema": {
      "description": "\nAppend input `B` to the end of input `A`.\n\n- It is required that this operation run in-place, meaning that the input `A` blob must match the output blob.\n- All except the outer-most dimension must be the same between `A` and `B`.\n- Input `A` may have to be re-allocated in order for accommodate to the new size. Currently, an exponential growth ratio is used in order to ensure amortized constant time complexity.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/dataset_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Append\",\n    [\"A\", \"B\"],\n    [\"A\"],\n)\n\nworkspace.FeedBlob(\"A\", np.random.randint(10, size=(1,3,3)))\nworkspace.FeedBlob(\"B\", np.random.randint(10, size=(2,3,3)))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"A:\", workspace.FetchBlob(\"A\"))\n\n```\n\n**Result**\n\n```\n\nA:\n[[[3 8 7]\n  [1 6 6]\n  [5 0 6]]]\nB:\n[[[4 3 1]\n  [7 9 6]\n  [9 4 5]]\n\n [[7 7 4]\n  [9 8 7]\n  [1 6 6]]]\nA:\n[[[3 8 7]\n  [1 6 6]\n  [5 0 6]]\n\n [[4 3 1]\n  [7 9 6]\n  [9 4 5]]\n\n [[7 7 4]\n  [9 8 7]\n  [1 6 6]]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor*): base input tensor of shape $(N, d_1, d_2, ..., d_n)$",
          "name": "A"
        },
        {
          "description": "(*Tensor*): second input tensor of shape $(M, d_1, d_2, ..., d_n)$ to be appended to the base",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor*): output tensor of shape $(N+M, d_1, d_2, ..., d_n)$",
          "name": "A"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AtomicIter",
    "schema": {
      "description": "\nSimilar to Iter, but takes a mutex as the first input to make sure that\nupdates are carried out atomically. This can be used in e.g. Hogwild sgd\nalgorithms.\n",
      "inputs": [
        {
          "description": "The mutex used to do atomic increment.",
          "name": "mutex"
        },
        {
          "description": "The iter counter as an int64_t TensorCPU.",
          "name": "iter"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StatRegistryUpdate",
    "schema": {
      "description": "\nUpdate the given StatRegistry, or the global StatRegistry,\nwith the values of counters for the given keys.\n",
      "inputs": [
        {
          "description": "1D string tensor with the key names to update.",
          "name": "keys"
        },
        {
          "description": "1D int64 tensor with the values to update.",
          "name": "values"
        },
        {
          "description": "If provided, update the given StatRegistry. Otherwise, update the global singleton.",
          "name": "handle"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "EnqueueRebatchingQueue",
    "schema": {
      "attributes": [
        {
          "description": "Are we enqueuing a batch or just a single element.         By default we enqueue single element.",
          "name": "enqueue_batch",
          "option": "optional"
        }
      ],
      "description": "\nEnqueues Tensors into the queue.\nNumber of input tensors should be equal to the number of components passed\nduring creation of the queue.\nIf the Queue is closed this operation will fail.\nIf enqueue_batch argument is set. We will split the input tensors by the\nfirst dimension to produce single queue elements.\n",
      "inputs": [
        {
          "description": "object representing the queue",
          "name": "queue"
        },
        {
          "description": "First tensor to enque. ",
          "name": "tensor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseAdam",
    "schema": {
      "attributes": [
        {
          "description": "Default 0.9",
          "name": "beta1",
          "option": "optional"
        },
        {
          "description": "Default 0.999",
          "name": "beta2",
          "option": "optional"
        },
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        },
        {
          "description": "Default false",
          "name": "enableRAdam",
          "option": "optional"
        }
      ],
      "description": "\n\n    Computes the Adam Update for the sparse case.\n    Given inputs (param, moment1, moment2, indices, grad, lr, iter), runs the dense\n    Adam on (param, moment1[indices], momemnt2[indices], lr, iter) and returns\n    (new_param, new_moment1, new_moment2) as in dense case.\n    Adam can be customized as Rectified Adam (RAdam) by setting enableRAdam = true.\n\n    ",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "First moment history",
          "name": "moment_1"
        },
        {
          "description": "Second moment history",
          "name": "moment_2"
        },
        {
          "description": "Sparse indices",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        },
        {
          "description": "iteration number",
          "name": "iter"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated first moment",
          "name": "output_moment_1"
        },
        {
          "description": "Updated second moment",
          "name": "output_moment_2"
        },
        {
          "description": "Optional Effective gradient",
          "name": "output_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "NormalizePlanarYUV",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "EnsureCPUOutput",
    "schema": {
      "description": "\nThis Op always create TensorCPU output, and may involves cross-device MemCpy.\nUnder CPU Context, this Op takes TensorCPU as input. Under the CUDA Context,\nthis Op accepts either CUDA or CPU Tensor input.\n",
      "inputs": [
        {
          "description": "The input CUDA or CPU tensor.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "TensorCPU that is a copy of the input.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FindDuplicateElements",
    "schema": {
      "description": "\nThe *FindDuplicateElements* op takes a single 1-D tensor *data* as input and returns a single 1-D output tensor *indices*. The output tensor contains the indices of the duplicate elements of the input, excluding the first occurrences. If all elements of *data* are unique, *indices* will be empty.\n\nGithub Links:\n\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/find_duplicate_elements_op.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/find_duplicate_elements_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"FindDuplicateElements\",\n    [\"data\"],\n    [\"indices\"],\n)\n\nworkspace.FeedBlob(\"data\", np.array([8,2,1,1,7,8,1]).astype(np.float32))\nprint(\"data:\\n\", workspace.FetchBlob(\"data\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"indices: \\n\", workspace.FetchBlob(\"indices\"))\n\n```\n\n**Result**\n\n```\n\ndata:\n [8. 2. 1. 1. 7. 8. 1.]\nindices:\n [3 5 6]\n\n```\n\n</details>\n\n\n  ",
      "inputs": [
        {
          "description": "a 1-D tensor.",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "Indices of duplicate elements in data, excluding first occurrences.",
          "name": "indices"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BernoulliJSDGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Conv1D",
    "schema": {
      "description": "\nThe convolution operator consumes an input vector, a 1D filter blob\nand a bias blob and computes the output. \nThe Conv2D operator computes a 2D convolution operation over an input blob $(X)$, with a filter blob $(filter)$ and a bias blob $(bias)$, and outputs a single output blob $(Y)$. Although there are several options for order, the convention is that the input $(X)$ is a blob of shape $(N,C_{in},H_{in},W_{in})$ and the output $(Y)$ is a blob of shape $(N,C_{out},H_{out},W_{out})$. Here, $N$ is the batch size, $C$ is the number of channels, $H$ is the spatial height, and $W$ is the spatial width. For example, if your input data was a batch of five, 100x120pixel RGB images, $X$ would have shape $(5,3,120,100)$.\n\nThe $filter$ input blob may contain multiple filters and has shape $(M, C_{in}, K_H, K_W)$. Here, $M$ is the number of individual filters contained in the blob, $C_{in}$ is the number of channels of each filter (by convention in 2D convolution it is the same as the number of channels in the input), $K_H$ is the spatial height of the kernel, and $K_W$ is the spatial width of the kernel. The $bias$ blob is a vector of length $M$, where there is one bias for each filter in the $filter$ blob.\n\nGiven the shape of the input blob and the filter blob, we can calculate the shape of the output blob as follows. The number of items in the batch $N$ will stay the same. The number of channels in the output will equal the number of kernels in the filter blob, so $C_{out} = M.$ With stride and pad defined below, the spatial height and width of the output ($H_{out}$ and $W_{out}$) are calculated as\n\n$$H_{out} = \\left \\lfloor{\\frac{H_{in} - K_H + 2*pad}{stride}+1}\\right \\rfloor$$\n\n\n$$W_{out} = \\left \\lfloor{\\frac{W_{in} - K_W + 2*pad}{stride}+1}\\right \\rfloor$$\n\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Conv\",\n    [\"X\", \"filter\", \"bias\"],\n    [\"Y\"],\n    kernel=5,\n    pad=1,\n    stride=2\n)\n\n// Create X: (N,C,H,W)\ndata = np.random.randn(1,1,8,8).astype(np.float32)\nprint(\"Data shape: \",data.shape)\n\n// Create W: (M,C,Kh,Kw)\nfilters = np.random.randn(3,1,5,5).astype(np.float32)\nprint(\"Filter shape: \",filters.shape)\n\n// Create b: M\nbias = np.array([1.,1.,1.]).astype(np.float32)\nprint(\"Bias shape: \",bias.shape)\n\n// Put the inputs into the workspace\nworkspace.FeedBlob(\"X\", data)\nworkspace.FeedBlob(\"filter\", filters)\nworkspace.FeedBlob(\"bias\", bias)\n\n// Run the operator\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nData shape:  (1, 1, 8, 8)\nFilter shape:  (3, 1, 5, 5)\nBias shape:  (3,)\nY:\n [[[[  0.6406407    0.8620521    0.56461596]\n   [ -1.5042953   -0.79549205 -10.683343  ]\n   [ -0.5240259    3.4538248   -3.9564204 ]]\n\n  [[  0.6876496    4.8328524   -1.9525816 ]\n   [  1.2995434   -2.3895378    7.2670045 ]\n   [  3.9929862    1.8126237    5.4699917 ]]\n\n  [[  3.55949      4.7934155    0.76086235]\n   [  3.9588015   -1.3251319    4.413117  ]\n   [ -1.5296054   -1.4924102   -3.2552304 ]]]]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "Input data blob, of shape $(N, C_{in}, H_{in}, W_{in})$, to be convolved with the kernels in the filter blob.",
          "name": "X"
        },
        {
          "description": "The filter blob, of shape $(M, C_{in}, K_H, K_W)$, containing the filters to be convolved with the data.",
          "name": "filter"
        },
        {
          "description": "The bias blob, of length $M$, containing the biases for the convolution, one bias per filter.",
          "name": "bias"
        }
      ],
      "outputs": [
        {
          "description": "Output data blob, of shape $(N, C_{out}, H_{out}, W_{out})$, that contains the result of the convolution.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsPad",
    "schema": {
      "attributes": [
        {
          "description": "The value to pad the data",
          "name": "padding_value",
          "option": "optional"
        },
        {
          "description": "The target length of each segment",
          "name": "target_length",
          "option": "optional"
        }
      ],
      "description": "\nGiven DATA tensor of rank r >= 1, and LENGTHS tensor of rank 1, pad each\nsegment in DATA with `value`, so that each segment's length is `target_length`.\nIf will throw, if there is segment of length larger than `target_length`.\n\nExample:\n  DATA  = [\n      [2.3, 3.4],\n      [4.5, 5.7],\n      [6.8, 7.9],\n  ]\n  LENGTHS = [0, 1, 1, 1]\n  and target_length = 2, padding value = -1.0\n  OUTPUT = [\n    [-1.0, -1.0],\n    [-1.0, -1.0],\n    [2.3, 3.4],\n    [-1.0, -1.0],\n    [4.5, 5.7],\n    [-1.0, -1.0],\n    [6.8, 7.9],\n    [-1.0, -1.0],\n  ]\n",
      "inputs": [
        {
          "description": "Tensor of rank r >= 1. First dimension must be equal to the size of lengths",
          "name": "DATA"
        },
        {
          "description": "Tensor of int32 lengths of rank 1",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "Padded DATA tensor",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReadNextBatch",
    "schema": {
      "attributes": [
        {
          "description": "Number of top-level entries to read.",
          "name": "batch_size",
          "option": "optional"
        }
      ],
      "description": "\nRead the next batch of examples out of the given cursor and data blobs.\n\nInput(0) is a blob pointing to a TreeCursor, and\n[Input(1),... Input(num_fields)] a list of tensors containing the data for\neach field of the dataset.\n\nReadNextBatch is thread safe.\n",
      "inputs": [
        {
          "description": "A blob containing a pointer to the cursor.",
          "name": "cursor"
        },
        {
          "description": "First dataset field",
          "name": "dataset_field_0"
        }
      ],
      "outputs": [
        {
          "description": "Tensor containing the next batch for field 0.",
          "name": "field_0"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "IntegralImageGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Cast",
    "schema": {
      "attributes": [
        {
          "description": "Data type to which the elements of the input tensor are cast. Strictly must be one of the types from *DataType* enum in TensorProto.",
          "name": "to",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nCasts the elements of a given input tensor to a data type specified by the `to`\nargument and returns an output tensor of the same size in the converted type.\nThe `to` argument must be one of the data types specified in the *DataType*\nenum field in the TensorProto message (see below). If the `to` argument is not\nprovided or is not one of the enumerated types in *DataType*, Caffe2 throws an\nEnforce error.\n\nNOTE: Casting from strings is not supported, and casting to strings is only\nsupported on CPU.\n\nTensorProto *DataType* field:\n```\nmessage TensorProto {\n  ...\n  enum DataType {\n    UNDEFINED = 0;\n    FLOAT = 1;  // float\n    INT32 = 2;  // int\n    BYTE = 3;  // BYTE, when deserialized, is going to be restored as uint8.\n    STRING = 4;  // string\n    BOOL = 5;  // bool\n    UINT8 = 6;  // uint8_t\n    INT8 = 7;  // int8_t\n    UINT16 = 8;  // uint16_t\n    INT16 = 9;  // int16_t\n    INT64 = 10;  // int64_t\n    FLOAT16 = 12;  // at::Half\n    DOUBLE = 13;  // double\n  }\n```\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/cast_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Cast\",\n    [\"X\"],\n    [\"Y\"],\n    to=2\n)\n\nworkspace.FeedBlob(\"X\", (np.random.rand(3,3)).astype(np.float32)*10)\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX: [[9.436466   5.8529844  0.54932857]\n [1.1583444  2.9936118  0.22950427]\n [3.9143739  3.4040766  8.905341  ]]\nY: [[9 5 0]\n [1 2 0]\n [3 3 8]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor)* Input tensor to be cast.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<'to' type>`)* Output tensor with the same shape as input with type specified by the `to` argument.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Barrier",
    "schema": {
      "description": "\nDoes a barrier operation among the nodes.\n",
      "inputs": [
        {
          "description": "The common world.",
          "name": "comm_world"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseToDense",
    "schema": {
      "description": "\nConvert sparse representations to dense with given indices.\n\nTransforms a sparse representation of map<id, value> represented as `indices`\nvector and `values` tensor into a compacted tensor where the first dimension\nis determined by the first dimension of the 3rd input if it is given or the\nmax index. Missing values are filled with zeros.\n\nThe op supports duplicated indices and performs summation over corresponding\nvalues. This behavior is useful for converting GradientSlices into dense\nrepresentation.\n\nAfter running this op:\n\n  output[indices[i], :] += values[i]  // sum over all indices[i] equal to the index\n  output[j, ...] = 0 if j not in indices\n",
      "inputs": [
        {
          "description": "1-D int32/int64 tensor of concatenated ids of data",
          "name": "indices"
        },
        {
          "description": "Data tensor, first dimension has to match `indices`, basic numeric types are supported",
          "name": "values"
        },
        {
          "description": "Optional: if provided, the first dimension of output is the first dimension of this tensor.",
          "name": "data_to_infer_dim"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor of the same type as `values` of shape `[len(lengths), len(mask)] + shape(default_value)` (if `lengths` is not provided the first dimension is omitted)",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Iter",
    "schema": {
      "description": "\nStores a singe integer, that gets incremented on each call to Run().\nUseful for tracking the iteration count during SGD, for example.\n",
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsMean8BitsRowwise",
    "schema": {
      "description": "\nVariation of SparseLengthsMean operator, where DATA is\nstored using 8bits. DATA was quantized with 8Bit row-wise\nquantization (see doc to FloatToRowwiseQuantized8Bits operator). To\nrestore DATA from 8Bit, we use additional input that stores scales\nand biases.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToRowwiseQuantized8Bits",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Matrix of floats, each row r_i of which stores a pair s_i, b_i -- scale and bias for i-th row",
          "name": "scale_bias"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LE",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise less or equal than comparison **<=** (with limited broadcast support).\n\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LE\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", np.array([1, 5, 2, 9, 12, 3]))\nworkspace.FeedBlob(\"B\", np.array([1, 3, 4, 9, 12, 8]))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\n\nA: [ 1  5  2  9 12  3]\nB: [ 1  3  4  9 12  8]\nC: [ True False  True  True  True  True]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<bool>`)* First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<bool>`)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<bool>`)* Output tensor with same dimensions as `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Split",
    "schema": {
      "attributes": [
        {
          "description": "(*int*): axis to split on",
          "name": "axis",
          "option": "optional"
        },
        {
          "description": "Pass non-zero integer to remove the axis specified in `axis` to all input tensors.",
          "name": "add_axis",
          "option": "optional",
          "type": "int64"
        },
        {
          "description": "(*Tuple(int)*): length of each output",
          "name": "split",
          "option": "optional"
        },
        {
          "description": "(*string*): order of dimensions of input and output blobs; either \"NCHW\" or \"NHWC\"",
          "name": "order",
          "option": "optional"
        }
      ],
      "description": "\nSplit an `input` tensor into a list of tensors, along the axis specified by the `axis` dimension. The lengths of the split can be specified using argument `split` or optional second input blob to the operator. Otherwise, the tensor is split to equal sized parts.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/concat_split_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Split\",\n    [\"input\"],\n    [\"output_0\",\"output_1\",\"output_2\"],\n    split=(3,2,4),\n    axis=0\n)\n\nworkspace.FeedBlob(\"input\", np.random.randint(10, size=(9)))\nprint(\"input:\", workspace.FetchBlob(\"input\"))\nworkspace.RunOperatorOnce(op)\nprint(\"output_0:\", workspace.FetchBlob(\"output_0\"))\nprint(\"output_1:\", workspace.FetchBlob(\"output_1\"))\nprint(\"output_2:\", workspace.FetchBlob(\"output_2\"))\n\n```\n\n**Result**\n\n```\n\ninput: [2 2 6 6 6 0 5 7 4]\noutput_0: [2 2 6]\noutput_1: [6 6]\noutput_2: [0 5 7 4]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor*): tensor to split",
          "name": "input"
        },
        {
          "description": "(*Tensor`<int>`*): [OPTIONAL] list of output lengths (see also arg `split`)",
          "name": "split"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor*): output tensor",
          "name": "[output_0, output_1, ...]"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsToWeights",
    "schema": {
      "attributes": [
        {
          "description": "n of 1/pow(length,n) for normalization",
          "name": "power",
          "option": "optional"
        }
      ],
      "description": "\nSimilar as LengthsToSegmentIds but output vector of segment\nweights derived by lengths. i.e 1/pow(length, power)\n",
      "inputs": [
        {
          "description": "1-D int32_t or int64_t tensor of lengths",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "1-D float tensor of weights by length",
          "name": "a vector of weights"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CreateAtomicBool",
    "schema": {
      "description": "Create an unique_ptr blob to hold an atomic<bool>",
      "outputs": [
        {
          "description": "Blob containing a unique_ptr<atomic<bool>>",
          "name": "atomic_bool"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RsqrtGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "GatherRanges",
    "schema": {
      "description": "\nGiven DATA tensor of rank 1, and RANGES tensor of rank 3, gather\ncorresponding ranges into a 1-D tensor OUTPUT.\n\nRANGES dimentions description:\n1: represents list of examples within a batch\n2: represents list features\n3: two values which are start and length or a range (to be applied on DATA)\n\nAnother output LENGTHS represents each example length within OUTPUT\n\nExample:\n  DATA  = [1, 2, 3, 4, 5, 6]\n  RANGES = [\n    [\n      [0, 1],\n      [2, 2],\n    ],\n    [\n      [4, 1],\n      [5, 1],\n    ]\n  ]\n  OUTPUT = [1, 3, 4, 5, 6]\n  LENGTHS = [3, 2]\n",
      "inputs": [
        {
          "description": "Tensor of rank 1.",
          "name": "DATA"
        },
        {
          "description": "Tensor of int32/int64 ranges, of dims (N, M, 2). Where N is number of examples and M is a size of each example. Last dimension represents a range in the format (start, lengths)",
          "name": "RANGES"
        }
      ],
      "outputs": [
        {
          "description": "1-D tensor of size sum of range lengths",
          "name": "OUTPUT"
        },
        {
          "description": "1-D tensor of size N with lengths over gathered data for each row in a batch. sum(LENGTHS) == OUTPUT.size()",
          "name": "LENGTHS"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CrossEntropy",
    "schema": {
      "description": "\nThis operator computes the cross entropy between a $NxD$ dimensional input data tensor $X$  and a $NxD$ dimensional input label tensor $label$. The op produces a single length $N$ output tensor $Y$. Here, $N$ is considered the batch size and $D$ is the size of each element in the batch. In practice, it is most commonly used at the end of models as a part of the loss computation, after the SoftMax operator and before the AveragedLoss operator. The cross entropy operation is defined as follows\n\n$$Y_i = \\sum_j (label_{ij} * log(X_{ij}))$$\n\nwhere ($i$, $j$) is the classifier's prediction of the $j$th class (the correct one), and $i$ is the batch size. Each log has a lower limit for numerical stability.\n\nGithub Links:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/cross_entropy_op.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/cross_entropy_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"CrossEntropy\",\n    [\"X\", \"label\"],\n    [\"Y\"]\n)\n\n// Create X: Sample softmax output for 5-class model\nX = np.array([[.01, .05, .02, .02, .9],[.03, .1, .42, .05, .4]])\nprint(\"X:\\n\",X)\n\n// Create label: Sample 1-hot ground truth label vectors\nlabel = np.array([[0.,0.,0.,0.,1.],[0.,0.,1.,0.,0.]])\nprint(\"label:\\n\",label)\n\n// Feed X & label into workspace\nworkspace.FeedBlob(\"X\", X.astype(np.float32))\nworkspace.FeedBlob(\"label\", label.astype(np.float32))\n\n// Run op\nworkspace.RunOperatorOnce(op)\n\n// Collect Output\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[0.01 0.05 0.02 0.02 0.9 ]\n [0.03 0.1  0.42 0.05 0.4 ]]\nlabel:\n [[0. 0. 0. 0. 1.]\n [0. 0. 1. 0. 0.]]\nY:\n [0.10536055 0.8675006 ]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "Input tensor which is almost always the result of a softmax operation. $X$ is a 2D array of size $NxD$, where $N$ is the batch size and $D$ is the number of classes.",
          "name": "X"
        },
        {
          "description": "Blob containing the labels used to compare the input. $label$ is the same shape as $X$.",
          "name": "label"
        }
      ],
      "outputs": [
        {
          "description": "Output blob from the cross entropy computation. $Y$ is 1D length $N$ tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceBackMaxGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ReduceBackMean",
    "schema": {
      "attributes": [
        {
          "description": "(*int*): number of dimensions to reduce (default=1)",
          "name": "num_reduce_dims",
          "option": "optional"
        }
      ],
      "description": "\nReduces the input tensor along the last dimension of the by applying **mean**.\n\nCan reduce more than one of the \"last\" dimensions by setting `num_reduce_dim`.\n\nA second (optional) input, `lengths`, can be passed, which enforces that only a subset of the elements are considered in the mean operation.\n- If input tensor `X` has shape $(d_0, d_1, d_2, ..., d_n)$, `lengths` must have shape $(d_0 * d_1 * d_2 * ... * d_{n-1})$.\n- The values of the `lengths` tensor determine how many of the values to consider for each vector in the $d_{n-1}$ dimension.\n\nFor example if $X = [[1,5,2,9],[4,1,8,2],[2,7,0,3]]$ and $lengths = [2,3,1]$, then $Y = [mean(1,5), mean(4,1,8), mean(2)] = [3, 4.333, 2]$\n\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduce_front_back_mean_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ReduceBackMean\",\n    [\"X\"],\n    [\"Y\"],\n    num_reduce_dim=2\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(1,2,3,3)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[[[5. 9. 0.]\n   [8. 4. 0.]\n   [2. 2. 4.]]\n\n  [[9. 0. 9.]\n   [7. 9. 7.]\n   [1. 0. 2.]]]]\nY: [[3.7777777 4.888889 ]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor",
          "name": "X"
        },
        {
          "description": "(*Tensor`<int>`*): number of elements in each sample",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): reduced tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ResizeNearest",
    "schema": {
      "attributes": [
        {
          "description": "Scale along width dimension",
          "name": "width_scale",
          "option": "optional"
        },
        {
          "description": "Scale along height dimension",
          "name": "height_scale",
          "option": "optional"
        }
      ],
      "description": "\nResizes the spatial dimensions of the input using nearest neighbor\ninterpolation. The `width_scale` and `height_scale` arguments\ncontrol the size of the output, which is given by:\noutput_width = floor(input_width * width_scale)\noutput_height = floor(output_height * height_scale)\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "X"
        },
        {
          "description": "1D, 2-element, Scales tensor, [height_scale, width_scale]",
          "name": "scales"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseDropoutWithReplacement",
    "schema": {
      "attributes": [
        {
          "default": 0.0,
          "description": "Probability of an element to be replaced.",
          "name": "ratio",
          "option": "optional",
          "type": "float32"
        },
        {
          "default": 0,
          "description": "Value elements are replaced with.",
          "name": "replacement_value",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\n\n`SparseDropoutWithReplacement` takes a 1-d input tensor and a lengths tensor.\nValues in the Lengths tensor represent how many input elements consitute each\nexample in a given batch.  The set of input values for an example will be\nreplaced with the single dropout value with probability given by the `ratio`\nargument.\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"SparseDropoutWithReplacement\",\n    [\"X\", \"Lengths\"],\n    [\"Y\", \"OutputLengths\"],\n    ratio=0.5,\n    replacement_value=-1\n)\n\nworkspace.FeedBlob(\"X\", np.array([1, 2, 3, 4, 5]).astype(np.int64))\nworkspace.FeedBlob(\"Lengths\", np.array([2, 3]).astype(np.int32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nprint(\"Lengths:\", workspace.FetchBlob(\"Lengths\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\nprint(\"OutputLengths:\", workspace.FetchBlob(\"OutputLengths\"))\n```\n\n**Result**\n\n```\nX: [1, 2, 3, 4, 5]\nLengths: [2, 3]\nY: [1, 2, -1]\nOutputLengths: [2, 1]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<int64_t>`)* Input data tensor.",
          "name": "X"
        },
        {
          "description": "*(type: Tensor`<int32_t>`)* Lengths tensor for input.",
          "name": "Lengths"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<int64_t>`)* Output tensor.",
          "name": "Y"
        },
        {
          "description": "*(type: Tensor`<int32_t>`)* Output tensor.",
          "name": "OutputLengths"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseWngrad",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\nThis operator implement the optimization algorithm\nin https://arxiv.org/abs/1803.02865 by Wu, Ward and Bottou.\nGiven inputs (param, seq_b, indices, grad, lr), runs the dense WnGrad\nupdate on (param, grad, seq_b, lr), and returns (new_param,\nnew_seq_b) as in the dense case.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "seq_b history",
          "name": "seq_b"
        },
        {
          "description": "Sparse indices",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated seq_b",
          "name": "output_seq_b"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Find",
    "schema": {
      "attributes": [
        {
          "description": "Placeholder for items that are not found",
          "name": "missing_value",
          "option": "optional"
        }
      ],
      "description": "\nFinds elements of second input from first input,\noutputting the last (max) index for each query.\nIf query not find, inserts missing_value.\nSee IndexGet() for a version that modifies the index when\nvalues are not found.\n",
      "inputs": [
        {
          "description": "Index (integers)",
          "name": "index"
        },
        {
          "description": "Needles / query",
          "name": "query"
        }
      ],
      "outputs": [
        {
          "description": "Indices of the needles in index or 'missing value'",
          "name": "query_indices"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BatchBucketOneHot",
    "schema": {
      "description": "\nInput is a matrix tensor. Its first dimension is the batch\nsize. For each column, bucketize it based on the boundary values and then do\none hot encoding. The `lengths` specifies the number of boundary values for each\ncolumn. The final number of buckets is this number plus 1. This would also be\nthe expanded feature size. `boundaries` specifies all the boundary values.\nNote that each bucket is right-inclusive. That is, given boundary values\n[b1, b2, b3], the buckets are defined as (-int, b1], (b1, b2], (b2, b3], (b3, inf).\nFor example\n\n  data = [[2, 3], [4, 1], [2, 5]], lengths = [2, 3],\n  If boundaries = [0.1, 2.5, 1, 3.1, 4.5], then\n  output = [[0, 1, 0, 0, 1, 0, 0], [0, 0, 1, 1, 0, 0, 0], [0, 1, 0, 0, 0, 0, 1]]\n\n  If boundaries = [0.1, 2.5, 1, 1, 3.1], then\n  output = [[0, 1, 0, 0, 0, 1, 0], [0, 0, 1, 0, 1, 0, 0], [0, 1, 0, 0, 0, 0, 1]]\n\n",
      "inputs": [
        {
          "description": "input tensor matrix",
          "name": "data"
        },
        {
          "description": "the size is the same as the width of the `data`",
          "name": "lengths"
        },
        {
          "description": "bucket boundaries",
          "name": "boundaries"
        }
      ],
      "outputs": [
        {
          "description": "output matrix that expands each input column with one hot encodingbased on the bucketization",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PairWiseLossGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ExpandGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "BatchGather",
    "schema": {
      "description": "\nBatch gather operation, first dimension in DATA is the batch size.\nGiven DATA tensor of rank r >= 2, and INDICES tensor of rank q >= 1, gather\nentries of the second outer dimension (axis == 1) of DATA indexed by INDICES,\nand concatenate them in an output tensor of rank q + (r - 1).\n\nExample:\n  DATA  = [\n      [1.0, 1.2, 2.4, 4.5],\n      [2.3, 3.4, 3.6, 2.3],\n      [4.5, 5.7, 1.2, 4.5],\n  ]\n  INDICES = [0, 2]\n\n  OUTPUT = [\n      [1.0, 2.4],\n      [2.3, 3.6],\n      [4.5, 1.2],\n  ]\n",
      "inputs": [
        {
          "description": "Tensor of rank r >= 2.",
          "name": "DATA"
        },
        {
          "description": "Tensor of int32/int64 indices, of any rank q.",
          "name": "INDICES"
        }
      ],
      "outputs": [
        {
          "description": "Tensor of rank q + (r - 1).",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Float16ConstantFill",
    "schema": {
      "attributes": [
        {
          "description": "The value for the elements of the output tensor.",
          "name": "value",
          "option": "optional"
        },
        {
          "description": "The shape of the output tensor.",
          "name": "shape",
          "option": "optional"
        }
      ],
      "description": null,
      "outputs": [
        {
          "description": "Output tensor of constant values specified by 'value'",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseSortedSegmentWeightedSum",
    "schema": {
      "attributes": [
        {
          "description": "Produce also gradient for `weights`. For now it's only supported in `Lengths`-based operators",
          "name": "grad_on_weights",
          "option": "optional"
        }
      ],
      "description": "\nPulls in slices of the input tensor, groups them into segments and applies\n'WeightedSum' to each segment. Segments need to be sorted and contiguous. See also\nSparseUnsortedSegmentWeightedSum that doesn't have this requirement.\n\nThis op is basically Gather and SortedSegmentWeightedSum fused together.\n\nINDICES should contain integers in range 0..N-1 where N is the first dimension\nof DATA. INDICES represent which slices of DATA need to be pulled in.\n\nSEGMENT_IDS is a vector that maps each referenced slice of the DATA to a\nparticular group (segment). Values belonging to the same segment are aggregated\ntogether. SEGMENT_IDS should have the same dimension as INDICES.\n\nThe first dimension of the output is equal to the number of input segments,\ni.e. `SEGMENT_IDS[-1]+1`. Other dimensions are inherited from the input tensor.\n\nInput slices are first scaled by SCALARS and then summed element-wise. It doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor for the summation",
          "name": "DATA"
        },
        {
          "description": "Scalar multipliers for the input slices. Must be a vector with the length matching the number of slices",
          "name": "SCALARS"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same length as INDICES and values in the range 0..K-1 and in increasing order that maps each slice of DATA referenced by INDICES to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of K (the number of segments).",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "DequeueRebatchingQueue",
    "schema": {
      "attributes": [
        {
          "description": "Number of elements to dequeue. By default we dequeue one element.",
          "name": "num_elements",
          "option": "optional"
        }
      ],
      "description": "\nDequeue Tensors from the Queue.\nIf the Queue is closed this might return less elements than asked.\nIf num_elements > 1 the returned elements will be concatenated into one\ntensor per component.\n",
      "inputs": [
        {
          "description": "object representing the queue",
          "name": "rebatching_queue"
        },
        {
          "description": "First tensor to enqueue",
          "name": "tensor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StoreGet",
    "schema": {
      "attributes": [
        {
          "description": "alternative key for the blob (optional)",
          "name": "blob_name",
          "option": "optional"
        }
      ],
      "description": "\nGet a blob from a store. The key is the output blob's name. The key\ncan be overridden by specifying the 'blob_name' argument.\n",
      "inputs": [
        {
          "description": "unique_ptr<StoreHandler>",
          "name": "handler"
        }
      ],
      "outputs": [
        {
          "description": "data blob",
          "name": "data"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MergeMultiScalarFeatureTensors",
    "schema": {
      "description": "Merge given multi-feature tensors with scalar features into one.\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": ".lengths",
          "name": "in1_lengths"
        },
        {
          "description": ".keys",
          "name": "in1_keys"
        },
        {
          "description": ".values",
          "name": "in1_values"
        }
      ],
      "outputs": [
        {
          "description": ".lengths",
          "name": "out_lengths"
        },
        {
          "description": ".keys",
          "name": "out_keys"
        },
        {
          "description": ".values",
          "name": "out_values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReluNGradient",
    "schema": {
      "attributes": [
        {
          "description": "the cap of forward op output",
          "name": "n",
          "option": "optional"
        }
      ],
      "description": "\nReluGradient takes both Y and dY and uses this to update dX according to the\nchain rule and derivatives of the rectified linear function.\n",
      "support_level": "default"
    }
  },
  {
    "name": "PadImageGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Assert",
    "schema": {
      "attributes": [
        {
          "description": "(*string*): custom error message to be thrown when the input does not pass assertion",
          "name": "error_msg",
          "option": "optional"
        }
      ],
      "description": "\nTakes in a tensor of type *bool*, *int*, *long*, or *long long* and checks if all values are True when coerced into a boolean. In other words, for non-bool types this asserts that all values in the tensor are non-zero. If a value is False after coerced into a boolean, the operator throws an error. Else, if all values are True, nothing is returned. For tracability, a custom error message can be set using the `error_msg` argument.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/assert_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Assert\",\n    [\"A\"],\n    [],\n    error_msg=\"Failed assertion from Assert operator\"\n)\n\nworkspace.FeedBlob(\"A\", np.random.randint(10, size=(3,3)).astype(np.int32))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\ntry:\n    workspace.RunOperatorOnce(op)\nexcept RuntimeError:\n    print(\"Assertion Failed!\")\nelse:\n    print(\"Assertion Passed!\")\n\n```\n\n**Result**\n\n```\n\nA:\n[[7 5 6]\n [1 2 4]\n [5 3 7]]\nAssertion Passed!\n\n```\n\n</details>\n\n        ",
      "inputs": [
        {
          "description": "(*Tensor*): input tensor",
          "name": "X"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Reduce",
    "schema": {
      "attributes": [
        {
          "description": "(int, default 0) the root to run reduce into.",
          "name": "root",
          "option": "optional"
        }
      ],
      "description": "\nDoes a reduce operation from every node to the root node. Currently only\nSum is supported.\n",
      "inputs": [
        {
          "description": "The common world.",
          "name": "comm_world"
        },
        {
          "description": "A tensor to be reduced.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "The reduced result on root, not set for other nodes.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "TimerGet",
    "schema": {
      "description": "\nQueries the current time of a timer object in nanoseconds.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/stats_ops.cc\n\n    ",
      "inputs": [
        {
          "description": "(*Tensor`<ptr>`*): pointer to a timer object; obtained from **TimerBegin** op",
          "name": "timer"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<int64>`*): scalar containing time in nanoseconds",
          "name": "nanos"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8GivenTensorFill",
    "schema": {
      "attributes": [
        {
          "description": "Input array of type char(byte)",
          "name": "values",
          "option": "optional"
        },
        {
          "description": "Input tensor shape",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "\n    Creates quantized tensor of type char(byte) with scale and zero point info.\n",
      "outputs": [
        {
          "description": "An Int8TensorCPU with scale and zero point info",
          "name": "Tensor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsIndicesInGradientSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "DotProductWithPadding",
    "schema": {
      "attributes": [
        {
          "description": "the padding value for tensors with smaller dimension",
          "name": "pad_value",
          "option": "optional"
        },
        {
          "description": "whether to replicate the smaller tensor or not",
          "name": "replicate",
          "option": "optional"
        }
      ],
      "description": "\nGiven two input float tensors X, Y with different shapes and produces one\noutput float tensor of the dot product between X and Y. We currently support\ntwo kinds of strategies to achieve this. Before doing normal dot_product 1)\npad the smaller tensor (using pad_value) to the same shape as the other one.\n2) replicate the smaller tensor to the same shape as the other one. Note the\nfirst dimension of X, Y must be equal. Only the second dimension of X or Y\ncan be padded.\n",
      "inputs": [
        {
          "description": "1D or 2D input tensor",
          "name": "X"
        },
        {
          "description": "1D or 2D input tensor",
          "name": "Y"
        }
      ],
      "outputs": [
        {
          "description": "1D output tensor",
          "name": "Z"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MakeTwoClassGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "HardSigmoidGradient",
    "schema": {
      "description": "\nHardSigmoidGradient takes both Y and dY as well as an argument alpha and uses\nthis to update dX according to the chain rule and derivatives of the hard\nsigmoid function.\n",
      "support_level": "default"
    }
  },
  {
    "name": "IndexFreeze",
    "schema": {
      "description": "\nFreezes the given index, disallowing creation of new index entries.\nShould not be called concurrently with IndexGet.\n",
      "inputs": [
        {
          "description": "Pointer to an Index instance.",
          "name": "handle"
        }
      ],
      "outputs": [
        {
          "description": "The input handle.",
          "name": "handle"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Scale",
    "schema": {
      "attributes": [
        {
          "description": "(float, default 1.0) the scale to apply.",
          "name": "scale",
          "option": "optional"
        }
      ],
      "description": "\nScale takes one input data (Tensor) and produces one output data\n(Tensor) whose value is the input data tensor scaled element-wise.\n",
      "support_level": "default"
    }
  },
  {
    "name": "APMeter",
    "schema": {
      "attributes": [
        {
          "description": "(int32_t) indicates how many predictions should the op buffer. defaults to 1000",
          "name": "buffer_size",
          "option": "optional"
        }
      ],
      "description": "\nAPMeter computes Average Precision for binary or multi-class classification.\nIt takes two inputs: prediction scores P of size (n_samples x n_classes), and\ntrue labels Y of size (n_samples x n_classes). It returns a single float number\nper class for the average precision of that class.\n",
      "inputs": [
        {
          "description": "2-D tensor (Tensor<float>) of size (num_samples xnum_classes) containing prediction scores",
          "name": "predictions"
        },
        {
          "description": "2-D tensor (Tensor<float>) of size (num_samples) containing true labels for each sample",
          "name": "labels"
        }
      ],
      "outputs": [
        {
          "description": "1-D tensor (Tensor<float>) of size num_classes containing average precision for each class",
          "name": "AP"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Rowwise8BitQuantizedToFloat",
    "schema": {
      "description": "\nGiven uint8 tensor, quantized using 8bit row-wise\nquantization, and auxiliary scales and biases, this operator\nrestores float tensor in the following way. We take input 8bits tensor\nof size  (m_1, m_2, ..., m_n), n >= 2, reshape it  into matrix of size\n(m_1, m_2 x... x m_n). We compute element r_{ij} of output matrix as\nr_{ij} * s_i + b_i and after this we reshape this output matrix into\noutput tensor of size (m_1, m_2, ..., m_n).\n",
      "inputs": [
        {
          "description": "quantized_input",
          "name": "quantized_input"
        },
        {
          "description": "Matrix of floats, each row r_i of which stores a pair s_i, b_i -- scale and bias for i-th row",
          "name": "scale_bias"
        }
      ],
      "outputs": [
        {
          "description": null,
          "name": null
        },
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Mod",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Divisor of the modulo operation (must be >= 1).",
          "name": "divisor",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": false,
          "description": "If true, sign of output matches divisor, else if false, sign follows dividend.",
          "name": "sign_follow_divisor",
          "option": "optional",
          "type": "boolean"
        }
      ],
      "description": "\nElement-wise modulo operation. Each element in the output is the modulo result\nof the corresponding element in the input data. The divisor of the modulo is\nprovided by the `divisor` argument.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/mod_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Mod\",\n    [\"X\"],\n    [\"Y\"],\n    divisor=10\n)\n\nworkspace.FeedBlob(\"X\", (np.random.randint(100, size=(5,5))))\nprint(\"X before running op:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"X after running op:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX before running op:\n[[56 22 43 13 60]\n [ 4 55 58 10 45]\n [64 66  4  3 66]\n [10 36 47 52 78]\n [91  4 36 47 95]]\nX after running op:\n[[6 2 3 3 0]\n [4 5 8 0 5]\n [4 6 4 3 6]\n [0 6 7 2 8]\n [1 4 6 7 5]]\n\n ```\n\n </details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<int>`)* Input tensor with int32 or int64 data.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<int>`)* Output tensor of data with modulo operation applied.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StringPrefix",
    "schema": {
      "attributes": [
        {
          "description": "Maximum size of the prefix, in bytes.",
          "name": "length",
          "option": "optional"
        }
      ],
      "description": "\nComputes the element-wise string prefix of the string tensor.\nInput strings that are shorter than prefix length will be returned unchanged.\nNOTE: Prefix is computed on number of bytes, which may lead to wrong behavior\nand potentially invalid strings for variable-length encodings such as utf-8.\n",
      "inputs": [
        {
          "description": "Tensor of std::string.",
          "name": "strings"
        }
      ],
      "outputs": [
        {
          "description": "Tensor of std::string containing prefixes for each input.",
          "name": "prefixes"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsSumFused8BitRowwise",
    "schema": {
      "description": "\nPerforms the same operation as SparseLengthsSum, but operating on\n8-bit rowwise quantized matrices with fused storage (where each row\nstores quantized values, and then 4-byte scale and 4-byte bias).\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused8BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FileStoreHandlerCreate",
    "schema": {
      "attributes": [
        {
          "description": "base path used by the FileStoreHandler",
          "name": "path",
          "option": "optional"
        },
        {
          "description": "prefix for all keys used by this store",
          "name": "prefix",
          "option": "optional"
        }
      ],
      "description": "\nCreates a unique_ptr<StoreHandler> that uses the filesystem as backing\nstore (typically a filesystem shared between many nodes, such as NFS).\nThis store handler is not built to be fast. Its recommended use is for\nintegration tests and prototypes where extra dependencies are\ncumbersome. Use an ephemeral path to ensure multiple processes or runs\ndon't interfere.\n",
      "outputs": [
        {
          "description": "unique_ptr<StoreHandler>",
          "name": "handler"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AtomicFetchAdd",
    "schema": {
      "description": "\nGiven a mutex and two int32 scalar tensors, performs an atomic fetch add\nby mutating the first argument and adding it to the second input\nargument. Returns the updated integer and the value prior to the update.\n",
      "inputs": [
        {
          "description": "Blob containing to a unique_ptr<mutex>",
          "name": "mutex_ptr"
        },
        {
          "description": "Value to be mutated after the sum.",
          "name": "mut_value"
        },
        {
          "description": "Value to add to the first operand.",
          "name": "increment"
        }
      ],
      "outputs": [
        {
          "description": "Mutated value after sum. Usually same as input 1.",
          "name": "mut_value"
        },
        {
          "description": "Value of the first operand before sum.",
          "name": "fetched_value"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Tanh",
    "schema": {
      "description": "\nCalculates the hyperbolic tangent of the given input tensor element-wise. This\noperation can be done in an in-place fashion too, by providing the same input\nand output blobs.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/tanh_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Tanh\",\n    [\"X\"],\n    [\"X\"],\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(3, 3).astype(np.float32))\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\n\nworkspace.RunOperatorOnce(op)\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[ 2.032603   -2.3556721  -0.14955314]\n [ 0.39309832 -1.1020128  -0.92951244]\n [-0.62815386  0.21342885  1.4002231 ]]\n\nX:\n [[ 0.9662601  -0.982175   -0.14844811]\n [ 0.3740282  -0.8012209  -0.73036647]\n [-0.55677974  0.21024609  0.8853999 ]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "1-D input tensor",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "The hyperbolic tangent values of the input tensor, computed element-wise",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Python",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "QuantDecodeGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "PythonGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "LengthsWeightedSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Squeeze",
    "schema": {
      "attributes": [
        {
          "description": "List of dimensions of *data* to squeeze out.",
          "name": "dims",
          "option": "optional",
          "type": "int64[]"
        }
      ],
      "category": "Transform",
      "description": "\nThe *Squeeze* op removes single-dimensional entries from the shape of the input tensor *data,* and produces a single output tensor *squeezed*. The op also takes an argument *dims* with a list of dimensions to squeeze. If the same blob is provided as input and output, the operation is copy-free. This is the exact inverse operation of *ExpandDims* given the same *dims* argument.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/expand_squeeze_dims_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/expand_squeeze_dims_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Squeeze\",\n    [\"data\"],\n    [\"squeezed\"],\n    dims=[0,1],\n)\n\nworkspace.FeedBlob(\"data\", np.zeros((1,1,100,100)).astype(np.float32))\nprint(\"data.shape:\", workspace.FetchBlob(\"data\").shape)\n\nworkspace.RunOperatorOnce(op)\nprint(\"squeezed.shape:\", workspace.FetchBlob(\"squeezed\").shape)\n\n```\n\n**Result**\n\n```\n\ndata.shape: (1, 1, 100, 100)\nsqueezed.shape: (100, 100)\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "Input tensor of data to be operated on.",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "Reshaped tensor with same data as input.",
          "name": "squeezed"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "NE",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise not equal to comparison **!=** (with limited broadcast support).\n\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"NE\",\n    [\"A\",  \"B\"],\n    [\"C\"],\n)\n\nworkspace.FeedBlob(\"A\", np.array([1, 5, 2, 9, 12, 3]))\nworkspace.FeedBlob(\"B\", np.array([1, 3, 4, 9, 12, 8]))\nprint(\"A:\", workspace.FetchBlob(\"A\"))\nprint(\"B:\", workspace.FetchBlob(\"B\"))\nworkspace.RunOperatorOnce(op)\nprint(\"C:\", workspace.FetchBlob(\"C\"))\n\n```\n\n**Result**\n\n```\nA: [ 1  5  2  9 12  3]\nB: [ 1  3  4  9 12  8]\nC: [False  True  True False False  True]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<bool>`)* First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor`<bool>`)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<bool>`)* Output tensor with same dimensions as `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceSum",
    "schema": {
      "attributes": [
        {
          "description": "(*Tuple(int)*): list of axes to reduce",
          "name": "axes",
          "option": "optional"
        },
        {
          "description": "(*int*): set to 1 to keep the reduced dimension(s) (default=1), else set to 0 to not keep the reduced dimension(s)",
          "name": "keepdims",
          "option": "optional"
        }
      ],
      "description": "\nComputes the **sum** of the input tensor's elements along the provided `axes`. The resulting tensor has the same rank as the input if the `keepdims` argument equals 1 (default). If `keepdims` is set to 0, then the `axes` dimensions are pruned.\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduce_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ReduceSum\",\n    [\"X\"],\n    [\"Y\"],\n    axes=(0,1),\n    keepdims=0\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(1,2,5,5)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[[[5. 3. 7. 9. 5.]\n   [4. 5. 1. 8. 3.]\n   [1. 0. 9. 7. 6.]\n   [7. 5. 0. 3. 1.]\n   [6. 4. 4. 8. 3.]]\n\n  [[8. 9. 6. 7. 7.]\n   [5. 5. 4. 7. 0.]\n   [9. 7. 6. 6. 7.]\n   [7. 5. 2. 4. 2.]\n   [4. 5. 1. 9. 4.]]]]\nY:\n[[13. 12. 13. 16. 12.]\n [ 9. 10.  5. 15.  3.]\n [10.  7. 15. 13. 13.]\n [14. 10.  2.  7.  3.]\n [10.  9.  5. 17.  7.]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): reduced tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "PackSegments",
    "schema": {
      "attributes": [
        {
          "description": "The pre-defined max_length for the packed segments",
          "name": "max_length",
          "option": "optional"
        },
        {
          "description": "Padding number in the packed segments. Use true to pad     -infinity, otherwise pad zeros",
          "name": "pad_minf",
          "option": "optional"
        },
        {
          "description": "bool whether to return presence mask, false by default",
          "name": "return_presence_mask",
          "option": "optional"
        }
      ],
      "description": "Map N dim tensor to N+1 dim based on length blob. Sequences that     are shorter than the longest sequence are padded with zeros.",
      "inputs": [
        {
          "description": "1-d int/long tensor contains the length in each of the output.",
          "name": "lengths"
        },
        {
          "description": "N dim Tensor.",
          "name": "tensor"
        }
      ],
      "outputs": [
        {
          "description": "N + 1 dim Tensorwhere dim(1) is the max length, dim(0) is the batch size.",
          "name": "packed_tensor"
        },
        {
          "description": "2 dim boolean tensor, false where packed_tensor is padded, true otherwise.",
          "name": "presence_mask"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "UnPackRecords",
    "schema": {
      "attributes": [
        {
          "description": "List of strings representing the string names in the formatspecified in the doc for CreateTreeCursor.",
          "name": "fields",
          "option": "optional"
        }
      ],
      "description": "\nGiven a packed dataset (packed by the PackRecordsOp) and the `fields` argument\ndescribing the datasets schema, return the original dataset format. Number of\nreturned tensors is equal to the number of fields in the `fields` argument.\n\nThe first input is the packed tensor to be unpacked. Optionally, you can provide\nprototype tensors to give the expected shapes of the output tensors. This is\nhelpful when you expected to unpack empty tensor, e.g., output of a sampling\nprocess.\n",
      "inputs": [
        {
          "description": "The tensor to be unpacked",
          "name": "packed_tensor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Normalize",
    "schema": {
      "attributes": [
        {
          "description": "axis to normalize",
          "name": "axis",
          "option": "optional"
        }
      ],
      "description": "\nGiven a matrix, apply L2-normalization along the specified dimension.\n",
      "support_level": "default"
    }
  },
  {
    "name": "LengthsMax",
    "schema": {
      "description": "\nApplies 'Max' to each segment of the input tensor. Segments are defined\nby their *LENGTHS*. *LENGTHS* is a vector that maps each of the slices of\n*DATA* to a particular segment. Values belonging to the same segment are\naggregated together and considered for the 'Max' operation.\n\nFor example *LENGTHS = [2, 1]* stands for segments *DATA[0..1]* and *DATA[2]*\n\nThe sum of elements in *LENGTHS* must equal the number of elements in the first\ndimension of *DATA*. The length of *OUTPUT* is equal to the number of input\nsegments, i.e. len(*LENGTHS*).\n\nMax computes the element-wise max of the input slices. Operation doesn't change the shape of the individual blocks.\n\n\nThe *LengthsMax* op takes two inputs *DATA* and *LENGTHS*, and produces a single output *OUTPUT*. The op finds the maximum value in each of the segments of *DATA*, where segments are defined by their lengths.\nFor example, if $DATA = [2,4,3,1,2,10]$ and $LENGTHS = [2,3,1]$ then $OUTPUT = [max([2,4]), max([3,1,2]), max([10])] = [4,3,10]$.\n\nGithub Link:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/segment_reduction_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LengthsMax\",\n    [\"DATA\", \"LENGTHS\"],\n    [\"OUTPUT\"],\n)\n\nworkspace.FeedBlob(\"DATA\", np.array([2,4,3,1,2,10]).astype(np.float32))\nprint(\"DATA:\\n\", workspace.FetchBlob(\"DATA\"))\n\nworkspace.FeedBlob(\"LENGTHS\", np.array([2,3,1]).astype(np.int32))\nprint(\"LENGTHS:\\n\", workspace.FetchBlob(\"LENGTHS\"))\n\nworkspace.RunOperatorOnce(op)\nprint(\"OUTPUT: \\n\", workspace.FetchBlob(\"OUTPUT\"))\n\n```\n\n**Result**\n\n```\n\nDATA:\n [ 2.  4.  3.  1.  2. 10.]\nLENGTHS:\n [2 3 1]\nOUTPUT:\n [ 4.  3. 10.]\n\n```\n\n</details>\n\n\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of len(LENGTHS) ",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "L1DistanceGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "MomentumSGD",
    "schema": {
      "description": "\n\nComputes a momentum SGD update for an input gradient and momentum\nparameters. Concretely, given inputs (grad, m, lr) and parameters\n(momentum, nesterov), computes:\n\n    if not nesterov:\n        adjusted_gradient = lr * grad + momentum * m\n        return (adjusted_gradient, adjusted_gradient)\n    else:\n        m_new = momentum * m + lr * grad\n        return ((1 + momentum) * m_new - momentum * m, m_new)\n\nOutput is (grad, momentum)\n\nNote the difference to MomemtumSGDUpdate, which actually performs the\nparameter update (and is thus faster).\n",
      "support_level": "default"
    }
  },
  {
    "name": "ReduceSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentRangeSum",
    "schema": {
      "description": "\nApplies 'Sum' to each segment of input tensor. In order to allow for more\nefficient implementation of 'Sum', the input segments have to be contiguous\nand non-empty.\n\nSEGMENT_IDS is a vector that maps each of the first dimension slices of the\nDATA to a particular group (segment). Values belonging to the same segment are\naggregated together.\n\nThe first dimension of the output is equal to the number of input segments,\ni.e. `SEGMENT_IDS[-1]+1`. Other dimensions are inherited from the input tensor.\n\nSummation is done element-wise across slices of the input tensor and doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor to be aggregated",
          "name": "DATA"
        },
        {
          "description": "Vector with the same length as the first dimension of DATA and values in the range 0..K-1 and in increasing order that maps each slice of DATA to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated tensor with the first dimension of K and the other dimentsions inherited from DATA",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "TT",
    "schema": {
      "attributes": [
        {
          "description": "(int[]) Input sizes of cores. Indicates the input size of the individual cores; the size of the input vector X must match the product of the inp_sizes array.",
          "name": "inp_sizes",
          "option": "optional"
        },
        {
          "description": "(int[]) Output sizes of cores. Indicates the output size of the individual cores; the size of the output vector Y must match the product of the out_sizes array.",
          "name": "out_sizes",
          "option": "optional"
        },
        {
          "description": "(int[]) Ranks of cores. Indicates the ranks of the individual cores; lower rank means larger compression, faster computation but reduce accuracy.",
          "name": "tt_ranks",
          "option": "optional"
        }
      ],
      "description": "\nThe TT-layer serves as a low-rank decomposition of a fully connected layer. The\ninputs are the same as to a fully connected layer, but the number of parameters\nare greatly reduced and forward computation time can be drastically reduced\nespecially for layers with large weight matrices. The multiplication is computed\nas a product of the input vector with each of the cores that make up the TT\nlayer. Given the input sizes (inp_sizes), output sizes(out_sizes), and the ranks\nof each of the cores (tt_ranks), the ith core will have size:\n\n    inp_sizes[i] * tt_ranks[i] * tt_ranks[i + 1] * out_sizes[i].\n\nThe complexity of the computation is dictated by the sizes of inp_sizes,\nout_sizes, and tt_ranks, where there is the trade off between accuracy of the\nlow-rank decomposition and the speed of the computation.\n",
      "inputs": [
        {
          "description": "Input tensor from previous layer with size (M x K), where M is the batch size and K is the input size.",
          "name": "X"
        },
        {
          "description": "1D blob containing the bias vector",
          "name": "b"
        },
        {
          "description": "1D blob containing each individual cores with sizes specified above.",
          "name": "cores"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor from previous layer with size (M x N), where M is the batch size and N is the output size.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BitwiseXor",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Pass 1 to enable broadcasting.",
          "name": "broadcast",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": -1,
          "description": "Axis to concatenate on. If set, defines the broadcast dimensions.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        }
      ],
      "description": "\nPerforms element-wise bitwise operation `bitwise_xor` (with limited broadcast support).\nBoth input operands should be of type `bool`.\n\nIf necessary the right-hand-side argument will be broadcasted to match the\nshape of left-hand-side argument. When broadcasting is specified, the second\ntensor can either be of size 1 (a scalar value), or having its shape as a\ncontiguous subset of the first tensor's shape. The starting of the mutually\nequal shape is specified by the argument \"axis\", and if it is not set, suffix\nmatching is assumed. 1-dim expansion doesn't work yet.\n\nFor example, the following tensor shapes are supported (with broadcast=1):\n```\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (5,)\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 4, 5), shape(B) = (2), with axis=0\n```\nArgument `broadcast=1` needs to be passed to enable broadcasting.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/elementwise_ops_schema.cc\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor)* First operand.",
          "name": "A"
        },
        {
          "description": "*(type: Tensor)* Second operand. With broadcasting can be of smaller size than `A`. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor)* Output tensor. Has same dimensions as input `A`.",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StoreSet",
    "schema": {
      "attributes": [
        {
          "description": "alternative key for the blob (optional)",
          "name": "blob_name",
          "option": "optional"
        }
      ],
      "description": "\nSet a blob in a store. The key is the input blob's name and the value\nis the data in that blob. The key can be overridden by specifying the\n'blob_name' argument.\n",
      "inputs": [
        {
          "description": "unique_ptr<StoreHandler>",
          "name": "handler"
        },
        {
          "description": "data blob",
          "name": "data"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CTCGreedyDecoder",
    "schema": {
      "attributes": [
        {
          "description": "When merge_repeated is true, merge repeated classes in output.",
          "name": "merge_repeated",
          "option": "optional"
        }
      ],
      "description": "Greedy decoder for connectionist temporal classification.",
      "inputs": [
        {
          "description": "3D float Tensor sized [max_time, batch_size, num_classes]",
          "name": "INPUTS"
        },
        {
          "description": "(optional) 1D int vector containing sequence lengths, having size [batch_size]seq_len will be set to max_time if not provided",
          "name": "SEQ_LEN"
        }
      ],
      "outputs": [
        {
          "description": "Output_len matrix size (batch). The row store: [decoded_length]",
          "name": "OUTPUT_LEN"
        },
        {
          "description": "Values vector, size (total_decoded_outputs). The vector stores the decoded classes",
          "name": "VALUES"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SegmentOneHot",
    "schema": {
      "description": "\nGiven a sequence of indices, segmented by the lengths tensor, returns a matrix\nthat has the elements in each sequence set to 1.0, and 0.0 everywhere else.\n",
      "inputs": [
        {
          "description": "Size of each segment.",
          "name": "lengths"
        },
        {
          "description": "Active indices, of size sum(lengths)",
          "name": "indices"
        },
        {
          "description": "Size of the index",
          "name": "index_size_tensor"
        }
      ],
      "outputs": [
        {
          "description": "Matrix of size len(lengths) x index_size",
          "name": "one_hots"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Max",
    "schema": {
      "description": "\nElement-wise max of an arbitrary number of input tensors. This operation can be\nperformed in-place, by using the first input blob as the output blob. All inputs\nmust have the same shape and data type, and the output will have the same shape\nas the inputs.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/minmax_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Max\",\n    [\"X\", \"Y\", \"Z\"],\n    [\"X\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.rand(3,3)).astype(np.float32))\nworkspace.FeedBlob(\"Y\", (np.random.rand(3,3)).astype(np.float32))\nworkspace.FeedBlob(\"Z\", (np.random.rand(3,3)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\nprint(\"Z:\", workspace.FetchBlob(\"Z\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Max:\", workspace.FetchBlob(\"X\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[0.4496477  0.07061381 0.7139333 ]\n [0.83203    0.05970785 0.72786295]\n [0.75988126 0.04601283 0.32820013]]\nY:\n[[0.05683139 0.16872478 0.671098  ]\n [0.70739156 0.09878621 0.03416285]\n [0.34087983 0.94986707 0.67263436]]\nZ:\n[[0.48051122 0.07141234 0.85264146]\n [0.77086854 0.22082241 0.13154659]\n [0.42401117 0.995431   0.4263775 ]]\nMax:\n[[0.48051122 0.16872478 0.85264146]\n [0.83203    0.22082241 0.72786295]\n [0.75988126 0.995431   0.67263436]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<Ord>`)* List of input tensors with the same shape.",
          "name": "X, Y, ..."
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<Ord>`)* Output tensor with same dimensions as input(s).Contains the maximum valued element at each location.",
          "name": "M"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MaxPool3DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CreateDB",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Rsqrt",
    "schema": {
      "description": "Computes the element-wise rsqrt of the input.",
      "inputs": [
        {
          "description": "ND input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "ND output tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "WeightedSum",
    "schema": {
      "description": "\nElement-wise weighted sum of several data, weight tensor pairs.\nInput should be in the form X_0, weight_0, X_1, weight_1, ... where X_i all\nhave the same shape, and weight_i are size 1 tensors that specifies the weight\nof each vector. Note that if one wants to do in-place computation, it could\nonly be done with X_0 also as the output, but not other X_i.\n",
      "inputs": [
        {
          "description": "Weight of the first input in the sum.",
          "name": "weight_0"
        }
      ],
      "outputs": [
        {
          "description": "Result containing weighted elem-wise sum of inputs.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "NCHW2NHWC",
    "schema": {
      "description": "\nThe operator switches the order of data in a tensor from NCHW- sample index N,\nchannels C, height H and width W, to the NHWC order (this is for 2D images).\nIn general, this operator switches the order of data in a tensor from N C H_1\n... H_k to N H_1 ... H_k C for k-dimensional features, and currently supports\nk=1, 2, and 3.\n",
      "inputs": [
        {
          "description": "The input data (Tensor) in the NCHW order.",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "The output tensor (Tensor) in the NHWC order.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GivenTensorByteStringToUInt8Fill",
    "schema": {
      "attributes": [
        {
          "description": "The value for the elements of the output tensor.",
          "name": "values"
        },
        {
          "description": "The shape of the output tensor.Cannot set the shape argument and pass in an input at the same time.",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "The additional dimensions appended at the end of the shape indicatedby the input blob.Cannot set the extra_shape argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional"
        },
        {
          "description": "1D tensor containing the desired output shape. First input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": "\nThis op fills a uint8 output tensor with the data specified by the *value* argument. The data must previously be serialized as a byte string. The output tensor shape is specified by the *shape* argument. Beware, when using this argument *value* should have a value for every element of the *output*, as missing values will not be initialized automatically. If *input_as_shape* is set to *true*, then the *input* should be a 1D tensor containing the desired output shape (the dimensions specified in *extra_shape* will also be appended). In this case, the *shape* argument should **not** be set.\n\nThis op allows us to write uint8 tensors to Protobuf as byte strings and read them back as uint8 tensors in order to avoid the Protobuf uint32_t varint encoding size penalty.\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nval = np.array([1, 2, 3], dtype=np.uint8)\nop = core.CreateOperator(\n    \"GivenTensorByteStringToUInt8Fill\",\n    [],\n    [\"out\"],\n    values=[val.tobytes()],\n    shape=val.shape,\n)\n\nworkspace.RunOperatorOnce(op)\nprint(\"Out:\\n\", workspace.FetchBlob(\"out\"))\n\n```\n\n**Result**\n\n```\n\nOut:\n [1 2 3]\n\n```\n\n</details>\n\n",
      "support_level": "default"
    }
  },
  {
    "name": "LC3DGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "rnn_internal_accumulate_gradient_input",
    "schema": {
      "description": "\nInternal RNN operator.\n",
      "support_level": "default"
    }
  },
  {
    "name": "SumInt",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "DotProductGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "MergeMultiMapFeatureTensors",
    "schema": {
      "description": "Merge given multi-feature tensors with map features into one.\n  Single-feature representation:\n  - scalar features:\n    <feature full name> T\n  - list features:\n    <feature full name>.lengths int32\n    <feature full name>.values T\n  - map features:\n    <feature full name>.lengths int32\n    <feature full name>.keys K\n    <feature full name>.values V\n\n  Missing values are set to zero, and value presence flag is set accordingly:\n    <feature full name>.presence bool\n\n  Multi-feature representation:\n  - scalar features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values T\n  - list features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.values T\n  - map features:\n    <feature type>.lengths int32\n    <feature type>.keys int64\n    <feature type>.values.lengths int32\n    <feature type>.values.keys K\n    <feature type>.values.values V\n\n  You can read more about representing batches of lists and maps here:\n  https://our.intern.facebook.com/intern/dex/caffe2/sparse-operations/\n",
      "inputs": [
        {
          "description": ".lengths",
          "name": "in1_lengths"
        },
        {
          "description": ".keys",
          "name": "in1_keys"
        },
        {
          "description": ".values.lengths",
          "name": "in1_values_lengths"
        },
        {
          "description": ".values.keys",
          "name": "in1_values_keys"
        },
        {
          "description": ".values.values",
          "name": "in1_values_values"
        }
      ],
      "outputs": [
        {
          "description": ".lengths",
          "name": "out_lengths"
        },
        {
          "description": ".keys",
          "name": "out_keys"
        },
        {
          "description": ".values_lengths",
          "name": "out_values_lengths"
        },
        {
          "description": ".values.keys",
          "name": "out_values_keys"
        },
        {
          "description": ".values.values",
          "name": "out_values_values"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SortedSegmentRangeLogSumExpGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "CosineSimilarityGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SoftmaxWithLossGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "EnforceFinite",
    "schema": {
      "description": "\nRaise if there is NaN or Inf values in the input tensor.\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "input"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AveragePool3D",
    "schema": {
      "description": "AveragePool3D \nconsumes an input blob and applies average pooling across the the blob according\nto kernel sizes, stride sizes, pad lengths and dilation. Average pooling consists\nof taking the average value of a subset of the input tensor according to the kernel\nsize and downsampling the data into the output blob for further processing. The\n`brew` module has a wrapper for this operator for use in a `ModelHelper` object.\n\nPooling layers reduce the spatial dimensionality of the input blob. Each of the\noutput blob's dimensions will reduce according to:\n\n$$dim_{out}=\\frac{dim_{in}-kernel+2*pad}{stride}+1$$\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"AveragePool\",\n    [\"X\"],\n    [\"Y\"],\n    kernel=2,\n    stride=2,\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(1, 1, 6, 6).astype(np.float32)) // NCHW\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX:\n [[[[-0.2883434   0.43498734  0.05417408  1.912558    0.09390241\n    -0.33173105]\n   [ 1.633709    1.2047161   0.36964908  0.99961185  0.4184147\n     0.9989975 ]\n   [ 1.7644193   0.1789665   1.5812988  -0.6038542  -0.36090398\n     0.33195344]\n   [ 0.9457722  -0.95174325 -0.78124577  1.2062047   1.1903144\n     0.2586746 ]\n   [ 1.252104    0.32645547  1.8073524  -0.78397465  0.9978303\n    -0.97614396]\n   [ 0.5440196   1.5778259  -0.76750124  0.5051756   0.8838398\n    -0.37085298]]]]\n\nY:\n [[[[0.7462672  0.83399826 0.2948959 ]\n   [0.4843537  0.3506009  0.35500962]\n   [0.9251013  0.19026303 0.13366827]]]]\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor of shape NCHW or NHWC.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output data tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceFrontSum",
    "schema": {
      "attributes": [
        {
          "description": "(*int*): number of dimensions to reduce (default=1)",
          "name": "num_reduce_dims",
          "option": "optional"
        }
      ],
      "description": "\nReduces the input tensor along the last dimension of the by applying **sum**.\n\nCan reduce more than one of the \"first\" dimensions by setting `num_reduce_dim`.\n\nA second (optional) input, `lengths`, can be passed, which enforces that only a subset of the elements are considered in the sum operation.\n- If input tensor `X` has shape $(d_0, d_1, d_2, ..., d_n)$, `lengths` must have shape $(d_1 * d_2 * ... * d_{n})$.\n- The values of the `lengths` tensor determine how many of the values to consider for each vector in the $d_{0}$ dimension.\n\nFor example, if $X = [[1,5,2,9],[4,1,8,2],[2,7,0,3]]$ and $lengths = [2,3,1,2]$, then $Y = [sum(1,4), sum(5,1,7), sum(2), sum(9,2)] = [2.5, 4.333, 2, 5.5]$\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduce_front_back_sum_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ReduceFrontSum\",\n    [\"X\"],\n    [\"Y\"],\n    num_reduce_dim=2\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(2,3,3)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[[4. 1. 1.]\n  [0. 6. 7.]\n  [7. 8. 6.]]\n\n [[5. 7. 7.]\n  [0. 1. 6.]\n  [2. 9. 0.]]]\nY: [18. 32. 27.]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor",
          "name": "X"
        },
        {
          "description": "(*Tensor`<int>`*): number of elements in each sample",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): reduced tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ConditionalSetAtomicBool",
    "schema": {
      "description": "\nSet an atomic<bool> to true if the given condition bool variable is true\n    ",
      "inputs": [
        {
          "description": "Blob containing a unique_ptr<atomic<bool>>",
          "name": "atomic_bool"
        },
        {
          "description": "Blob containing a bool",
          "name": "condition"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CollectTensor",
    "schema": {
      "attributes": [
        {
          "description": "The max number of tensors to collect",
          "name": "num_to_collect",
          "option": "optional"
        }
      ],
      "description": "\nCollect tensor into tensor vector by reservoir sampling,\nargument num_to_collect indicates the max number of tensors that will be\ncollected. The first half of the inputs are tensor vectors, which are also the\noutputs. The second half of the inputs are the tensors to be collected into each\nvector (in the same order). The input tensors are collected in all-or-none\nmanner. If they are collected, they will be placed at the same index in the\noutput vectors.\n",
      "support_level": "default"
    }
  },
  {
    "name": "Min",
    "schema": {
      "description": "\nElement-wise min of an arbitrary number of input tensors. This operation can be performed in-place, by using the first input blob as the output blob. All inputs must have the same shape and data type, and the output will have the same shape as the inputs.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/minmax_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Min\",\n    [\"X\", \"Y\", \"Z\"],\n    [\"X\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.rand(2,2)).astype(np.float32))\nworkspace.FeedBlob(\"Y\", (np.random.rand(2,2)).astype(np.float32))\nworkspace.FeedBlob(\"Z\", (np.random.rand(2,2)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\nprint(\"Z:\", workspace.FetchBlob(\"Z\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Min:\", workspace.FetchBlob(\"X\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[0.32731926 0.4939747 ]\n [0.29242373 0.43460014]]\nY:\n[[0.40928316 0.916115  ]\n [0.77526504 0.29339448]]\nZ:\n[[0.7899794  0.90335774]\n [0.82599413 0.2843068 ]]\nMin:\n[[0.32731926 0.4939747 ]\n [0.29242373 0.2843068 ]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<Ord>`)* List of input tensors with the same shape.",
          "name": "X, Y, ..."
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<Ord>`)* Output tensor with same dimensions as input(s).Contains the minimum valued element at each location.",
          "name": "M"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Ceil",
    "schema": {
      "description": "\nElement-wise application of the ceil function ($y=ceil(x)$) to the input tensor\n`X`. Output tensor shape is the same as the input tensor.\n\nGithub Link:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/ceil_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Ceil\",\n    [\"X\"],\n    [\"X\"],\n)\n\nworkspace.FeedBlob(\"X\", (np.random.uniform(-10, 10, (5,5))).astype(np.float32))\nprint(\"X before running op:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"X after running op:\", workspace.FetchBlob(\"X\"))\n\n```\n\n**Result**\n\n```\n\nX before running op:\n[[ 8.44598    -6.5098248  -2.2993476  -7.6859694   0.58566964]\n [-7.846551   -0.03689406  6.9362907  -4.0521703   4.4969673 ]\n [ 0.33355865 -7.895527   -8.393201    9.374202   -2.3930092 ]\n [-6.3061996   3.1403487   3.782099   -8.516556   -2.8387244 ]\n [-2.0164998   4.7663913  -3.422966    0.3636999   8.75713   ]]\nX after running op:\n[[ 9. -6. -2. -7.  1.]\n [-7. -0.  7. -4.  5.]\n [ 1. -7. -8. 10. -2.]\n [-6.  4.  4. -8. -2.]\n [-2.  5. -3.  1.  9.]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RecurrentNetworkBlobFetcher",
    "schema": {
      "attributes": [
        {
          "description": "Prefix string to prepend extracted blobs.",
          "name": "prefix",
          "option": "optional"
        }
      ],
      "description": "\nRetrieves blobs from scratch workspaces (which contain intermediate recurrent\nnetwork computation for each timestep) and puts them in the global\nworkspace under CPUContext.\n",
      "inputs": [
        {
          "description": "Name of scratch workspace blob returned by recurrent network.",
          "name": "ScratchWorkspaceBlob"
        }
      ],
      "outputs": [
        {
          "description": "1D tensor of strings containing extracted blob names.",
          "name": "blob_names"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Selu",
    "schema": {
      "attributes": [
        {
          "default": 1.673263,
          "description": "Alpha constant in equation.",
          "name": "alpha",
          "option": "optional",
          "type": "float32"
        },
        {
          "default": 1.0507,
          "description": "Scale constant in equation.",
          "name": "scale",
          "option": "optional",
          "type": "float32"
        }
      ],
      "description": "\n\nThe *Selu* op takes one input tensor $X$, an argument $alpha$, an argument $scale$, and produces one output tensor $Y$ of the same shape as $X.$ The op performs the element wise *Selu* operation, defined as\n\n$$y=selu(x) =\\begin{cases}scale (\\alpha e^{x} - \\alpha) & x < 0\\\\scale * x & otherwise\\end{cases}$$\n\nThe default value of *alpha* is 1.6732632423543772848170429916717 and the default value of *scale* is 1.0507009873554804934193349852946. See [Self-Normalizing Neural Networks](https://arxiv.org/abs/1706.02515) for more information.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/selu_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/selu_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Selu\",\n    [\"X\"],\n    [\"Y\"],\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(3, 3).astype(np.float32))\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\n\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[ 1.1613879  -0.27111396 -1.2076733 ]\n [ 1.3442237  -1.0701777   1.2070968 ]\n [ 0.23810555  0.9740916  -1.7872391 ]]\n\nY:\n [[ 1.2202715  -0.4174965  -1.2326177 ]\n [ 1.4123772  -1.1551634   1.2682979 ]\n [ 0.25017774  1.023479   -1.4637551 ]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "Input tensor of data to be operated on.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor with same shape as input.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "DataCouple",
    "schema": {
      "description": "\n\nA one to one operator that takes an arbitrary number of input and output blobs\nsuch that each input blob is inplace with it's matching output blob. It then proceedes\nto do nothing with each of these operators. This serves two purposes. It can make it\nappear as if a blob has been written to, as well as can tie together different blobs\nin a data dependency\n\n",
      "support_level": "default"
    }
  },
  {
    "name": "Logit",
    "schema": {
      "attributes": [
        {
          "description": "small positive epsilon value, the default is 1e-6.",
          "name": "eps (optional)",
          "option": "optional"
        }
      ],
      "description": "\nElementwise logit transform: logit(x) = log(x / (1 - x)), where x is the\ninput data clampped in (eps, 1-eps).\n",
      "inputs": [
        {
          "description": "input float tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "output float tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SegmentIdsToLengths",
    "schema": {
      "description": "\nTransfers a vector of segment ids to a vector of segment lengths. This operation\nsupports non-consecutive segment ids. Segments not appearing in the input vector\nwill have length 0. If the second input is provided, the number of segments =\nthe size of its first dimension. Otherwise, the number of segments = the last\nindex in the first input vector + 1.\n\nIn general, for consecutive, zero-based segment IDs, this is the inverse\noperation of LengthsToSegmentIds, except that a vector of segment IDs\ncannot represent empty segments at the end (if the second input is absent).\n",
      "inputs": [
        {
          "description": "1-D int32_t or int64_t tensor of segment ids",
          "name": "segment_ids"
        },
        {
          "description": "if provided, number of segments = the size of its first dimension",
          "name": "data (optional)"
        }
      ],
      "outputs": [
        {
          "description": "1-D int64_t tensor of segment lengths",
          "name": "lengths"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "YellowFin",
    "schema": {
      "attributes": [
        {
          "description": "Default 0.999",
          "name": "beta",
          "option": "optional"
        },
        {
          "description": "Default 20",
          "name": "curv_win_width",
          "option": "optional"
        },
        {
          "description": "Default 1e-6",
          "name": "epsilon",
          "option": "optional"
        },
        {
          "description": "Default false",
          "name": "nesterov",
          "option": "optional"
        },
        {
          "description": "Default true",
          "name": "zero_debias",
          "option": "optional"
        }
      ],
      "description": "\n\nComputes the YellowFin update (https://arxiv.org/abs/1706.03471) and performs\nmomentum SGD optimization step. lr and mu are not being shared between\nparameters. curv_win, g_avg, g2_avg and scalars_memory are just auxiliary\nmemory for computing moving averages (see the publication). Takes arguments\nbeta: coefficient for moving averages,\ncurv_win_width: timeframe when average squared gradient is being stored,\nepsilon: for numerical purposes,\nnesterov and zero_debias for debias of moving average.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Momentum",
          "name": "moment"
        },
        {
          "description": "Learning rate",
          "name": "lr"
        },
        {
          "description": "Momentum coefficient",
          "name": "mu"
        },
        {
          "description": "Memory for latest curvature ranges",
          "name": "curv_win"
        },
        {
          "description": "Moving average of gradient",
          "name": "g_avg"
        },
        {
          "description": "Moving average of squared gradient",
          "name": "g2_avg"
        },
        {
          "description": "Memory for stateful scalars",
          "name": "scalars_memory"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "Iteration number",
          "name": "iter"
        }
      ],
      "outputs": [
        {
          "description": "Parameters to be updated",
          "name": "output_param"
        },
        {
          "description": "Momentum",
          "name": "output_moment"
        },
        {
          "description": "Output learning rate",
          "name": "output_lr"
        },
        {
          "description": "Output momentum coefficient",
          "name": "output_mu"
        },
        {
          "description": "Output memory for latest curvature ranges",
          "name": "output_curv_win"
        },
        {
          "description": "Output moving average of gradient",
          "name": "output_g_avg"
        },
        {
          "description": "Output moving average of squared gradient",
          "name": "output_g2_avg"
        },
        {
          "description": "Output memory for stateful scalars",
          "name": "output_scalars_memory"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceBackMax",
    "schema": {
      "attributes": [
        {
          "description": "(*int*): number of dimensions to reduce (default=1)",
          "name": "num_reduce_dims",
          "option": "optional"
        }
      ],
      "description": "\nReduces the input tensor along the last dimension of the by applying **max**.\n\nCan reduce more than one of the \"last\" dimensions by setting `num_reduce_dim`.\n\nA second (optional) input, `lengths`, can be passed, which enforces that only a subset of the elements are considered in the max operation.\n- If input tensor `X` has shape $(d_0, d_1, d_2, ..., d_n)$, `lengths` must have shape $(d_0 * d_1 * d_2 * ... * d_{n-1})$.\n- The values of the `lengths` tensor determine how many of the values to consider for each vector in the $d_{n-1}$ dimension.\n\nFor example if $X = [[1,5,2,9],[4,1,8,2],[2,7,0,3]]$ and $lengths = [2,3,1]$, then $Y = [max(1,5), max(4,1,8), max(2)] = [5, 8, 2]$\n\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/reduce_front_back_max_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"ReduceBackMax\",\n    [\"X\"],\n    [\"Y\"],\n    num_reduce_dim=2\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(10, size=(1,2,3,3)).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n[[[[2. 5. 1.]\n   [6. 1. 9.]\n   [8. 5. 9.]]\n\n  [[5. 7. 8.]\n   [9. 9. 6.]\n   [6. 5. 0.]]]]\nY: [[9. 9.]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(*Tensor`<float>`*): input tensor",
          "name": "X"
        },
        {
          "description": "(*Tensor`<int>`*): number of elements in each sample",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor`<float>`*): reduced tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8Concat",
    "schema": {
      "attributes": [
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        },
        {
          "description": "Which axis to concat on",
          "name": "axis",
          "option": "optional"
        },
        {
          "description": "Pass 1 to add the axis specified in arg 'axis' to all input tensors",
          "name": "add_axis",
          "option": "optional"
        }
      ],
      "description": "Concatenate a list of tensors into a single tensor",
      "outputs": [
        {
          "description": "Concatenated tensor",
          "name": "concat_result"
        },
        {
          "description": "The dimensions of the inputs.",
          "name": "split_info"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Allgather",
    "schema": {
      "description": "\nDoes an allgather operation among the nodes.\n",
      "inputs": [
        {
          "description": "The common world.",
          "name": "comm_world"
        },
        {
          "description": "A tensor to be allgathered.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "The allgathered tensor, same on all nodes.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RecurrentNetworkGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "HalfFloatToFused8BitRowwiseQuantized",
    "schema": {
      "description": "\nApplies 8-bit row-wise quantization by determining the range\n(maximum - minimum) and offset (minimum value) of each row in the input\nmatrix, and then scaling each element to an 8-bit number between 0 and\n255. To later de-quantize values, the scale (range / 255) and offset\n(bias) are stored alongside the data. More precisely, each row contains\nint8 elements for each quantized element, and the last 8 bytes\nof each row in the output matrix are a float storing the scale\nfollowed by another float containing the scale.)\n",
      "inputs": [
        {
          "description": "Float16 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "StopGradient",
    "schema": {
      "description": "\nStopGradient is a helper operator that does no actual numerical computation,\nand in the gradient computation phase stops the gradient from being computed\nthrough it.\n",
      "support_level": "default"
    }
  },
  {
    "name": "Summarize",
    "schema": {
      "attributes": [
        {
          "description": "(int, default 0) flag to indicate if the summarized statistics have to be written to a log file.",
          "name": "to_file",
          "option": "optional"
        }
      ],
      "description": "\nSummarize computes four statistics of the input tensor (Tensor)- min,\nmax, mean and standard deviation. The output will be written to a 1-D tensor of\nsize 4 if an output tensor is provided. Else, if the argument 'to_file' is\ngreater than 0, the values are written to a log file in the root folder.\n",
      "inputs": [
        {
          "description": "The input data as Tensor.",
          "name": "data"
        }
      ],
      "outputs": [
        {
          "description": "1-D tensor (Tensor) of size 4 containing min, max, mean and standard deviation",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "NormalizeL1",
    "schema": {
      "attributes": [
        {
          "description": "axis to normalize",
          "name": "axis",
          "option": "optional"
        }
      ],
      "description": "\nGiven a matrix, apply L1-normalization along the specified axis.\n",
      "support_level": "default"
    }
  },
  {
    "name": "BatchSparseToDense",
    "schema": {
      "attributes": [
        {
          "description": "Optional, output dense last dimension. If both this argument and output_shape_inference are set, it should be consistent with output_shape_inference's last dim",
          "name": "dense_last_dim",
          "option": "optional"
        },
        {
          "description": "Optional, missing values are filled with this value.default_value = 0 when not set",
          "name": "default_value",
          "option": "optional"
        }
      ],
      "description": "\nConvert sparse matrix representation into dense matrix.\n\nA sparse matrix is represented by `lengths` vector, `indices` vector,\nand `values` vector. Each element in `lengths` vector (lengths[`i`]) represents\nthe number of indices in this batch (batch `i`).\nWith in each batch, `indices` should not have duplicate number.\n\nFor example, with input:\n\n  lengths = [2, 3, 1]\n  indices = [0, 1, 2, 3, 4, 5]\n  values =  [6, 7, 8, 9, 10, 11]\n  dense_dim = 6\n  default_value = 0\n\nThe output is:\n\n  output = [[6, 7, 0, 0, 0,  0],\n            [0, 0, 8, 9, 10, 0],\n            [0, 0, 0, 0, 0, 11]]\n\nafter running this operator.\n",
      "inputs": [
        {
          "description": "Flatten tensor, used to break down indices and values into per batch indices and values.",
          "name": "lengths"
        },
        {
          "description": "Flatten tensor of total size = \\sum lengths, containing the indices ",
          "name": "indices"
        },
        {
          "description": "Data tensor, dimension has to match `indices`",
          "name": "values"
        },
        {
          "description": "Optional, a dense tensor whose shape define the output shape",
          "name": "output_shape_inference"
        }
      ],
      "outputs": [
        {
          "description": "2-D dense tensor, with 1st dim = len(lengths), 2nd dim = dense_last_dimin the arg list, the tensor is of the same data type as `values`.Missing values are filled with default_value",
          "name": "dense"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MaxPool3D",
    "schema": {
      "description": "MaxPool3D \nconsumes an input blob and applies max pooling across the the blob according to\nkernel sizes, stride sizes, pad lengths and dilation. Max pooling consists of\ntaking the maximum value of a subset of the input tensor according to the kernel\nsize and downsampling the data into the output blob for further processing. The\n`brew` module has a wrapper for this operator for use in a `ModelHelper` object.\n\nPooling layers reduce the spatial dimensionality of the input blob. Each of the\noutput blob's dimensions will reduce according to:\n\n$$dim_{out}=\\frac{dim_{in}-kernel+2*pad}{stride}+1$$\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.h\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/pool_op.cc\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/conv_pool_op_base.h\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"MaxPool\",\n    [\"X\"],\n    [\"Y\"],\n    kernel=2,\n    stride=2,\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(1, 1, 6, 6).astype(np.float32)) // NCHW\nprint(\"X:\\n\", workspace.FetchBlob(\"X\"), \"\\n\")\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n```\n\n**Result**\n\n```\nX:\n [[[[-2.8534958e-01 -1.7719941e+00 -8.2277227e-04  1.1088650e+00\n    -2.1476576e+00 -3.5070452e-01]\n   [-9.0058845e-01 -3.0070004e-01 -1.7907504e+00 -7.1746534e-01\n     1.2798511e+00 -3.2214901e-01]\n   [ 1.5806322e+00  1.6845188e+00 -2.6633200e-01 -3.8576153e-01\n    -9.6424848e-02 -3.9696163e-01]\n   [ 1.2572408e-01  6.3612902e-01 -3.9554062e-01 -6.9735396e-01\n    -9.1898698e-01 -1.9609968e-01]\n   [-1.1587460e+00  2.4605224e+00 -1.5497679e+00  1.3020347e-01\n    -8.1293899e-01 -7.8803545e-01]\n   [ 1.4323474e+00  1.3618395e+00  9.8975077e-02 -1.1307785e-01\n     7.2035044e-01  2.7642491e-01]]]]\n\nY:\n [[[[-0.28534958  1.108865    1.2798511 ]\n   [ 1.6845188  -0.266332   -0.09642485]\n   [ 2.4605224   0.13020347  0.72035044]]]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input data tensor of shape NCHW or NHWC.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Output data tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "CreateMap",
    "schema": {
      "attributes": [
        {
          "description": "Key's TensorProto::DataType (default INT32)",
          "name": "key_dtype",
          "option": "optional"
        },
        {
          "description": "Value's TensorProto::DataType (default INT32)",
          "name": "value_dtype",
          "option": "optional"
        }
      ],
      "description": "Create an empty map blob",
      "outputs": [
        {
          "description": "Blob reference to the map",
          "name": "map blob"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BatchMomentsGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "WeightedSigmoidCrossEntropyWithLogitsGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "RmsProp",
    "schema": {
      "description": "\nComputes the RMSProp update\n(http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf).\nConcretely, given inputs (grad, mean_squares, mom, lr), computes:\n\n    mean_squares_o = mean_squares + (1 - decay) * (square(grad) - mean_squares)\n    mom_o = momentum * mom + lr * grad / sqrt(epsilon + mean_squares_o)\n    grad_o = mom_o\n\nReturns (grad_o, mean_squares_o, mom_o).\n",
      "support_level": "default"
    }
  },
  {
    "name": "SoftmaxWithLoss",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "Setting to 1 enables inputting labels as probability distribution.",
          "name": "label_prob",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": 1,
          "description": "Axis of the inputs when coerced to 2D.",
          "name": "axis",
          "option": "optional",
          "type": "int64"
        },
        {
          "description": "Average loss output scaling factor (must be >= 0).",
          "name": "scale",
          "option": "optional",
          "type": "float32"
        },
        {
          "default": "'NCHW'",
          "description": "Order of blob dimensions (only 'NCHW' is supported currently).",
          "name": "order",
          "option": "optional",
          "type": "string"
        }
      ],
      "description": "\nCombined Softmax and Cross-Entropy loss operator. The operator first computes the softmax normalized values for each layer in the batch of the given input, then computes cross-entropy loss. This operator is numerically more stable than separate `Softmax` and `CrossEntropy` ops. The inputs are a 2-D tensor `logits` of size (batch_size x input_feature_dimensions), which represents the unscaled log probabilities, and a 1-dimensional integer `labels` tensor for ground truth. An optional third input blob (`weight_tensor`) can be used to weight the samples for the loss, which is useful if the training set is unbalanced. This operator outputs a `softmax` tensor which contains the probability for each label for each example (same shape is `logits` input), and a scalar `loss` value, which is the averaged cross-entropy loss between the softmax probabilities and the ground truth values. Use parameter `label_prob`=1 to enable inputting labels as a probability distribution.\n\nSoftmax cross-entropy loss function:\n\n$$loss(x, class) = -\\log{\\biggl(\\frac{\\exp(x[class])}{\\sum_{j} \\exp(x[j])}\\biggr)} = -x[class] + \\log{\\biggl(\\sum_{j} \\exp(x[j])\\biggr)}$$\n\nor if the `weight_tensor` has been passed:\n\n$$loss(x, class) = weight[class]\\biggl(-x[class] + \\log{\\biggl(\\sum_{j} \\exp(x[j])\\biggr)}\\biggr)$$\n\nThe `logits` input does not need to explicitly be a 2D vector; rather, it will be coerced into one. For an arbitrary n-dimensional tensor `X` in $[a_0, a_1, ..., a_{k-1}, a_k, ..., a_{n-1}]$, where k is the `axis` provided, then `X` will be coerced into a 2-dimensional tensor with dimensions $[(a_0 * ... * a_{k-1}), (a_k * ... * a_{n-1})]$. For the default case where `axis`=1, the `X` tensor will be coerced into a 2D tensor of dimensions $[a_0, (a_1 * ... * a_{n-1})]$, where $a_0$ is often the batch size. In this situation, we must have $a_0 = N$ and $a_1 * ... * a_{n-1} = D$. Each of these dimensions must be matched correctly, or else the operator will throw errors.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/softmax_with_loss_op.cc\n\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"SoftmaxWithLoss\",\n    [\"logits\", \"labels\"],\n    [\"softmax\", \"avgloss\"]\n)\n\nworkspace.FeedBlob(\"logits\", np.random.randn(1, 5).astype(np.float32))\nworkspace.FeedBlob(\"labels\", np.asarray([4]).astype(np.int32))\nprint(\"logits:\", workspace.FetchBlob(\"logits\"))\nprint(\"labels:\", workspace.FetchBlob(\"labels\"))\nworkspace.RunOperatorOnce(op)\nprint(\"softmax:\", workspace.FetchBlob(\"softmax\"))\nprint(\"avgloss:\", workspace.FetchBlob(\"avgloss\"))\n\n```\n\n**Result**\n\n```\n\nlogits: [[-0.3429451  -0.80375195  0.23104447  1.4569176  -0.5268362 ]]\nlabels: [4]\nsoftmax: [[0.09721052 0.0613179  0.17258129 0.58800864 0.0808817 ]]\navgloss: 2.5147676\n\n```\n\n</details>\n\n<details>\n\n<summary> <b>Example 2</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"SoftmaxWithLoss\",\n    [\"logits\", \"labels\"],\n    [\"softmax\", \"avgloss\"],\n    scale=5.0\n)\n\nworkspace.FeedBlob(\"logits\", np.asarray([[.1, .4, .7, 1.5, .2]]).astype(np.float32))\nworkspace.FeedBlob(\"labels\", np.asarray([4]).astype(np.int32))\nprint(\"logits:\", workspace.FetchBlob(\"logits\"))\nprint(\"labels:\", workspace.FetchBlob(\"labels\"))\nworkspace.RunOperatorOnce(op)\nprint(\"softmax:\", workspace.FetchBlob(\"softmax\"))\nprint(\"avgloss:\", workspace.FetchBlob(\"avgloss\"))\n\n```\n\n**Result**\n\n```\n\nlogits: [[0.1 0.4 0.7 1.5 0.2]]\nlabels: [4]\nsoftmax: [[0.10715417 0.144643   0.19524762 0.4345316  0.11842369]]\navgloss: 10.667433\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* Input tensor.",
          "name": "logits"
        },
        {
          "description": "*(type: Tensor`<float>`)* Ground truth label tensor.",
          "name": "labels"
        },
        {
          "description": "*(type: Tensor`<float>`)* [OPTIONAL] Blob used to weight the samples for the loss.",
          "name": "weight_tensor"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Softmax output tensor.",
          "name": "softmax"
        },
        {
          "description": "*(type: float)* Averaged cross-entropy loss output.",
          "name": "loss"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "VariableLengthSequencePadding",
    "schema": {
      "description": "\nSuper special-case operator. Used to pad a tensor to mimic pytorch's\npad_packed_sequence.\n\nGiven an input tensor INPUT of size NxBxM and an input tensor LENS\nof size B, where\n\nN = maximum sequence length\nB = batch size\nM = hidden size\n\nset each element of INPUT to zero if it is is past the end of the\ncorresponding sequence (i.e. if LENS[j] > i for an index (i,j,k)).\n\n",
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SoftplusGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsMean",
    "schema": {
      "description": "\nPulls in slices of the input tensor, groups them into segments and applies\n'Mean' to each segment. Segments are defined by their LENGTHS.\n\nThis op is basically Gather and LengthsMean fused together.\n\nINDICES should contain integers in range 0..N-1 where N is the first dimension\nof DATA. INDICES represent which slices of DATA need to be pulled in.\n\nLENGTHS is a vector that defines slice sizes by first dimension of DATA. Values\nbelonging to the same segment are aggregated together. sum(LENGTHS) has\nto match INDICES size.\n\nThe first dimension of the output is equal to the number of input segment,\ni.e. `len(LENGTHS)`. Other dimensions are inherited from the input tensor.\n\nMean computes the element-wise mean of the input slices. Operation doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Non negative vector with sum of elements equal to INDICES length",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of K (the number of segments).",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseUnsortedSegmentWeightedSum",
    "schema": {
      "attributes": [
        {
          "description": "Produce also gradient for `weights`. For now it's only supported in `Lengths`-based operators",
          "name": "grad_on_weights",
          "option": "optional"
        }
      ],
      "description": "\nPulls in slices of the input tensor, groups them into segments and applies\n'WeightedSum' to each segment. Segments ids can appear in arbitrary order (unlike in\nSparseSortedSegmentWeightedSum).\n\nThis op is basically Gather and UnsortedSegmentWeightedSum fused together.\n\nINDICES should contain integers in range 0..N-1 where N is the first dimension\nof DATA. INDICES represent which slices of DATA need to be pulled in.\n\nSEGMENT_IDS is a vector that maps each referenced slice of the DATA to a\nparticular group (segment). Values belonging to the same segment are aggregated\ntogether. SEGMENT_IDS should have the same dimension as INDICES.\n\nIf `num_segments` argument is passed it would be used as a first dimension for\nthe output. Otherwise, it'd be dynamically calculated from as the max value of\nSEGMENT_IDS plus one. Other output dimensions are inherited from the input\ntensor.\n\nInput slices are first scaled by SCALARS and then summed element-wise. It doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor for the summation",
          "name": "DATA"
        },
        {
          "description": "Scalar multipliers for the input slices. Must be a vector with the length matching the number of slices",
          "name": "SCALARS"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Integer vector with the same length as INDICES that maps each slice of DATA referenced by INDICES to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of equal to the number of segments.",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "EnsureDense",
    "schema": {
      "description": "\nThis operator converts dense or sparse gradients to dense ones.\nTherefore, sparse gradient can be back propagated to Operators that consume\ndense gradients only (e.g., FCGradient).\n\nThe operator's behaviors:\n\n- In forward, simply pass in place or copy input to the output.\n- In backward, if the gradient passed-in is sparse gradient, change it to dense gradient in linear time; otherwise, simply pass the dense gradient.\n",
      "inputs": [
        {
          "description": "Input tensors.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor. Same dimension as inputs.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LabelCrossEntropy",
    "schema": {
      "description": "\nThis operator computes the cross entropy between a $NxD$ dimensional input data tensor $X$  and a one dimensional input label tensor $label$. The op produces a single length $N$ output tensor $Y$. Here, $N$ is considered the batch size and $D$ is the size of each element in the batch. In practice, it is most commonly used at the end of models as a part of the loss computation, after the SoftMax operator and before the AveragedLoss operator. The cross entropy operation is defined as follows\n\n$$Y_i = -log(X_{ij})$$\n\nwhere ($i$, $j$) is the classifier's prediction of the $j$th class (the correct one), and $i$ is the batch size. Each log has a lower limit for numerical stability.\n\nThe difference between *LabelCrossEntropy* and *CrossEntropy* is how the labels are specified. Here, the labels are a length $N$ list of integers, whereas in CrossEntropy the labels are a $NxD$ dimensional matrix of one hot label vectors. However, the results of computation should be the same, as shown in the two examples where ($i$, $j$) is the classifier's prediction of the $j$th class (the correct one), and $i$ is the batch size. Each log has a lower limit for numerical stability.\n\nGithub Links:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/cross_entropy_op.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/cross_entropy_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"LabelCrossEntropy\",\n    [\"X\", \"label\"],\n    [\"Y\"]\n)\n\n// Create X: Sample softmax output for 5-class model\nX = np.array([[.01, .05, .02, .02, .9],[.03, .1, .42, .05, .4]])\nprint(\"X:\\n\",X)\n\n// Create label: Sample 1-hot ground truth label vectors\nlabel = np.array([4,2])\nprint(\"label:\\n\",label)\n\n// Feed X & label into workspace\nworkspace.FeedBlob(\"X\", X.astype(np.float32))\nworkspace.FeedBlob(\"label\", label.astype(np.int32))\n\n// Run op\nworkspace.RunOperatorOnce(op)\n\n// Collect Output\nprint(\"Y:\\n\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX:\n [[0.01 0.05 0.02 0.02 0.9 ]\n [0.03 0.1  0.42 0.05 0.4 ]]\nlabel:\n [4 2]\nY:\n [0.10536055 0.8675006 ]\n\n```\n\n</details>\n\n\n",
      "inputs": [
        {
          "description": "Input tensor which is almost always the result of a softmax operation. $X$ is a 2D array of size $NxD$, where $N$ is the batch size and $D$ is the number of classes.",
          "name": "X"
        },
        {
          "description": "Blob containing the labels used to compare the input. $label$ is a length $N$ list of integers, where each element is the integer label for the $n$th element of the batch.",
          "name": "label"
        }
      ],
      "outputs": [
        {
          "description": "Output blob from the cross entropy computation. $Y$ is 1D length $N$ tensor.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BooleanMaskGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Save",
    "schema": {
      "attributes": [
        {
          "default": 0,
          "description": "If set to non-zero, save the db directly to the path specified by the `db` arg. If not set (default), prepend the path of the current root folder of the workspace to the path specified by the `db` arg.",
          "name": "absolute_path",
          "option": "optional",
          "type": "int64"
        },
        {
          "default": "",
          "description": "Characters in the provided blob names that match `strip_prefix` will be removed prior to saving. Also, characters that precede `strip_prefix` will be removed. Useful for removing device scope from blob names.",
          "name": "strip_prefix",
          "option": "optional",
          "type": "string"
        },
        {
          "description": "If set, used as blob names instead of original blob names. Must be same length as number of blobs.",
          "name": "blob_name_overrides",
          "option": "optional",
          "type": "string[]"
        },
        {
          "description": "The output path of the db. See the `absolute_path` arg details for options regarding the current root folder of the workspace.",
          "name": "db",
          "option": "optional",
          "type": "string"
        },
        {
          "description": "Type of db to save (options: \"lmdb\", \"leveldb\", \"minidb\").",
          "name": "db_type",
          "option": "optional",
          "type": "string"
        },
        {
          "default": "kDefaultChunkSize",
          "description": "The chunk size to split tensor data into. If not set, caffe2_tensor_chunk_size will be used",
          "name": "chunk_size",
          "option": "optional",
          "type": "string"
        }
      ],
      "description": "\nSaves a set of blobs to a db. It takes $[1, \\infty)$ number of inputs and has\nno output. The contents of the inputs are written into the db using the\nsettings specified by the arguments.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/load_save_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Save\",\n    [\"X\", \"Y\", \"Z\"],\n    [],\n    db=\"test_db2\",\n    db_type=\"leveldb\",\n    blob_name_overrides=[\"x_scores\", \"y_scores\", \"z_scores\"]\n)\n\nworkspace.FeedBlob(\"X\", np.random.randint(20, size=(5,5)))\nworkspace.FeedBlob(\"Y\", np.random.randint(20, size=(5,5)))\nworkspace.FeedBlob(\"Z\", np.random.randint(20, size=(5,5)))\nworkspace.RunOperatorOnce(op)\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor)* Input tensor(s).",
          "name": "X"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseSortedSegmentMean",
    "schema": {
      "description": "\nPulls in slices of the input tensor, groups them into segments and applies\n'Mean' to each segment. Segments need to be sorted and contiguous. See also\nSparseUnsortedSegmentMean that doesn't have this requirement.\n\nThis op is basically Gather and SortedSegmentMean fused together.\n\nINDICES should contain integers in range 0..N-1 where N is the first dimension\nof DATA. INDICES represent which slices of DATA need to be pulled in.\n\nSEGMENT_IDS is a vector that maps each referenced slice of the DATA to a\nparticular group (segment). Values belonging to the same segment are aggregated\ntogether. SEGMENT_IDS should have the same dimension as INDICES.\n\nThe first dimension of the output is equal to the number of input segments,\ni.e. `SEGMENT_IDS[-1]+1`. Other dimensions are inherited from the input tensor.\n\nMean computes the element-wise mean of the input slices. Operation doesn't change the shape of the individual blocks.\n  ",
      "inputs": [
        {
          "description": "Input tensor, slices of which are aggregated.",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same length as INDICES and values in the range 0..K-1 and in increasing order that maps each slice of DATA referenced by INDICES to one of the segments",
          "name": "SEGMENT_IDS"
        }
      ],
      "outputs": [
        {
          "description": "Aggregated output tensor. Has the first dimension of K (the number of segments).",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "GaussianFill",
    "schema": {
      "attributes": [
        {
          "default": 0.0,
          "description": "Mean of the distribution to draw from.",
          "name": "mean",
          "option": "optional",
          "type": "float32"
        },
        {
          "default": 1.0,
          "description": "Standard deviation of the distribution to draw from.",
          "name": "std",
          "option": "optional",
          "type": "float32"
        },
        {
          "description": "Desired shape of the *output* tensor.",
          "name": "shape",
          "option": "optional",
          "type": "int64[]"
        },
        {
          "description": "The additional dimensions appended at the end of the *shape* indicated by the input blob. Cannot set the *extra_shape* argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional",
          "type": "int64[]"
        },
        {
          "default": false,
          "description": "set to *True* to use the *input* as shape. First, input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional",
          "type": "boolean"
        }
      ],
      "description": "\nThis op fills an output tensor with samples drawn from a normal distribution specified by the mean and standard deviation arguments. The output tensor shape is specified by the *shape* argument. However, if *input_as_shape* is set to *true*, then the *input* should be a 1D tensor containing the desired output shape (the dimensions specified in *extra_shape* will also be appended). In this case, the *shape* argument should **not** be set.\n\n*Note: cannot set the shape argument and pass in an input at the same time.*\n\nGithub Links:\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/filler_op.h\n- https://github.com/caffe2/caffe2/blob/master/caffe2/operators/filler_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"GaussianFill\",\n    [],\n    [\"out\"],\n    shape=[3,3],\n    mean=2.0,\n    std=1.1\n)\n\nworkspace.RunOperatorOnce(op)\nprint(\"Out:\\n\", workspace.FetchBlob(\"out\"))\n\n```\n\n**Result**\n\n```\n\nOut:\n [[1.2084167  2.3336504  2.827349  ]\n [2.7108908  0.9374752  1.7173369 ]\n [0.03320992 2.1775863  1.0894578 ]]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "(Optional) 1D tensor specifying the shape of the output. Must be used with *input_as_shape=True*",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor of random values drawn from a normal distribution. If the shape argument is set, this is the shape specified, and if the *input* exists and *input_as_shape=True*, it is the shape specified by the *input* tensor.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ReduceMinGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "GivenTensorInt16Fill",
    "schema": {
      "attributes": [
        {
          "description": "The value for the elements of the output tensor.",
          "name": "values"
        },
        {
          "description": "The shape of the output tensor.Cannot set the shape argument and pass in an input at the same time.",
          "name": "shape",
          "option": "optional"
        },
        {
          "description": "The additional dimensions appended at the end of the shape indicatedby the input blob.Cannot set the extra_shape argument when there is no input blob.",
          "name": "extra_shape",
          "option": "optional"
        },
        {
          "description": "1D tensor containing the desired output shape. First input must be in CPU context.",
          "name": "input_as_shape",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "Abs",
    "schema": {
      "description": "\nCalculates the absolute value of the given input tensor, element-wise.\n\nGithub Links:\n\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/abs_op.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"Abs\",\n    [\"X\"],\n    [\"Y\"]\n)\n\nworkspace.FeedBlob(\"X\", np.random.randn(5).astype(np.float32))\nprint(\"X:\", workspace.FetchBlob(\"X\"))\nworkspace.RunOperatorOnce(op)\nprint(\"Y:\", workspace.FetchBlob(\"Y\"))\n\n```\n\n**Result**\n\n```\n\nX: [ 0.3005476   1.551666   -1.3591481   0.39191285 -0.21866608]\nY: [0.3005476  1.551666   1.3591481  0.39191285 0.21866608]\n\n```\n\n</details>\n\n",
      "inputs": [
        {
          "description": "*(type: Tensor<float>)* Input tensor.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "*(type: Tensor`<float>`)* Absolute value of input element-wise.",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsTopK",
    "schema": {
      "attributes": [
        {
          "description": "the number of top values to return for each segment, if the number of values is smaller than k, the values would be padded with 0 and indices would be padded with -1.",
          "name": "k",
          "option": "optional"
        }
      ],
      "description": "\nApply TopK to each segment of the input tensor, where segments are defined by\ntheir LENGTHS, and concatenate them in an output tensor of\nshape=(SIZE(LENGTHs), k). In case there's less than k values in a segment,\nthe output value will be padded by 0, and the corresponding output indices will\nbe padded by -1.\n",
      "inputs": [
        {
          "description": "Tensor of rank 1. First dimension must be equal to the sum of lengths",
          "name": "DATA"
        },
        {
          "description": "Tensor of int32 lengths of rank 1",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "Output top k elements for each segment, withshape=(SIZE(lengths), k)",
          "name": "TopKValue"
        },
        {
          "description": "Output indices in DATA corresponding to value in TopKValue",
          "name": "TopKIndices"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RMACRegions",
    "schema": {
      "attributes": [
        {
          "description": "Number of scales to sample regions at.",
          "name": "scales",
          "option": "optional"
        },
        {
          "description": "Overlap between consecutive regions.",
          "name": "overlap",
          "option": "optional"
        }
      ],
      "description": "\nComputes a fixed-grid of RMAC region coordinates at various levels\nas described in https://arxiv.org/abs/1511.05879.\n",
      "inputs": [
        {
          "description": "The input 4D tensor of shape NCHW.",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "The output RMAC regions for all items in the batch. Tensor of shape (N x 5) following the ROIPoolOp format - each row is of the format (batch_index x1 y1 x2 y2) where x1, y1, x2, y2 are the region co-ordinates. Each region is repeated N times corresponding to each item in the batch.",
          "name": "RMAC_REGIONS"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SumReduceLike",
    "schema": {
      "attributes": [
        {
          "description": "If set, defines the starting dimension for reduction. Args `axis` and `axis_str` cannot be used simultaneously.",
          "name": "axis",
          "option": "optional"
        },
        {
          "description": "If set, it could only be N or C or H or W. `order` arg should also be provided. It defines the reduction dimensions on NCHW or NHWC. Args `axis` and `axis_str` cannot be used simultaneously.",
          "name": "axis_str",
          "option": "optional"
        },
        {
          "description": "Either NHWC or HCWH",
          "name": "order",
          "option": "optional"
        }
      ],
      "description": "\nSumReduceLike operator takes 2 tensors as input. It performs reduce sum to the\nfirst input so that the output looks like the second one.\nIt assumes that the first input\nhas more dimensions than the second, and the dimensions of the second input is\nthe contiguous subset of the dimensions of the first.\nFor example, the following tensor shapes are supported:\n\n  shape(A) = (2, 3, 4, 5), shape(B) = (4, 5)\n  shape(A) = (2, 3, 4, 5), shape(B) = (,), i.e. B is a scalar\n  shape(A) = (2, 3, 4, 5), shape(B) = (3, 4), with axis=1\n  shape(A) = (2, 3, 2, 5), shape(B) = (2), with axis=0\n    ",
      "inputs": [
        {
          "description": "First operand, should share the type with the second operand.",
          "name": "A"
        },
        {
          "description": "Second operand. With broadcasting can be of smaller size than A. If broadcasting is disabled it should be of the same size.",
          "name": "B"
        }
      ],
      "outputs": [
        {
          "description": "Result, has same dimensions and type as B",
          "name": "C"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "MaxGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ReplaceNaN",
    "schema": {
      "attributes": [
        {
          "description": "the value to replace NaN, the default is 0",
          "name": "value (optional)",
          "option": "optional"
        }
      ],
      "description": "\nReplace the NaN (not a number) element in the input tensor with argument `value`\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "input"
        },
        {
          "description": "Output tensor",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HasScope",
    "schema": {
      "description": "\nChecks whether scope blob has any saved scopes left\n    ",
      "support_level": "default"
    }
  },
  {
    "name": "ReduceBackSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ZeroGradient",
    "schema": {
      "description": "\nZeroGradient operators doesn't produce any output blobs. One can use\nthis operator to produce 0 gradient for the input blob.\n",
      "support_level": "default"
    }
  },
  {
    "name": "ReduceBackMeanGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ClipGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsWeightedSum8BitsRowwise",
    "schema": {
      "description": "\nVariation of SparseLengthsWeightedSum operator, where\nDATA is stored using 8bits. DATA was quantized with 8Bit row-wise\nquantization (see doc to FloatToRowwiseQuantized8Bits operator). To\nrestore DATA from 8Bit, we use additional input that stores scales\nand biases.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToRowwiseQuantized8Bits",
          "name": "DATA"
        },
        {
          "description": "Scalar multipliers for the input slices. Must be a vector with the length matching the length of INDICES",
          "name": "SCALARS"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Matrix of floats, each row r_i of which stores a pair s_i, b_i -- scale and bias for i-th row",
          "name": "scale_bias"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HuffmanTreeHierarchy",
    "schema": {
      "attributes": [
        {
          "description": "The number of classes used to build the hierarchy.",
          "name": "num_classes",
          "option": "optional"
        }
      ],
      "description": "\nHuffmanTreeHierarchy is an operator to generate huffman tree hierarchy given\nthe input labels. It returns the tree as serialized HierarchyProto\n",
      "inputs": [
        {
          "description": "The labels vector",
          "name": "Labels"
        }
      ],
      "outputs": [
        {
          "description": "Huffman coding hierarchy of the labels",
          "name": "Hierarch"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ChannelShuffle",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "AveragedLossGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsIndicesInGradientSumGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "BooleanUnmask",
    "schema": {
      "description": "\nGiven a series of masks and values, reconstruct values together according to masks. A comprehensive example:\n```\nmask1   = True, False, True, False, False\nvalues1 = 1.0, 3.0\nmask2   = False, True, False, False, False\nvalues2 = 2.0\nmask3   = False, False, False, True, True\nvalues3 = 4.0, 5.0\n```\n\nReconstruct by:\n\n```\noutput = net.BooleanUnmask([mask1, values1, mask2, values2, mask3, values3], [\"output\"])\noutput = 1.0, 2.0, 3.0, 4.0, 5.0\n```\n\nNote that for all mask positions, there must be at least one True. This is not allowed:\n\n```\nmask1   = True, False\nvalues1 = 1.0\nmask2   = False, False\nvalues2 =\n\noutput = net.BooleanUnmask([mask1, values1, mask2, values2], [\"output\"])\n```\n\nIf there are multiple True values for a field, we accept the first value, and no longer expect a value for that location:\n\n```\nmask1   = True, False\nvalues1 = 1.0\nmask2   = True, True\nvalues2 = 2.0\n\noutput = net.BooleanUnmask([mask1, values1, mask2, values2], [\"output\"])\noutput = 1.0, 2.0\n```\n\n*** Note that we alternate `data` and `mask` inputs\n\nGithub Links:\n- https://github.com/pytorch/pytorch/blob/master/caffe2/operators/boolean_unmask_ops.cc\n\n<details>\n\n<summary> <b>Example</b> </summary>\n\n**Code**\n\n```\n\nworkspace.ResetWorkspace()\n\nop = core.CreateOperator(\n    \"BooleanUnmask\",\n    [\"mask1\", \"data1\", \"mask2\", \"data2\"],\n    [\"unmasked_data\"]\n)\n\nworkspace.FeedBlob(\"mask1\", np.array([True,False,False,True,True,False]))\nworkspace.FeedBlob(\"data1\", np.array([1,4,5]))\nworkspace.FeedBlob(\"mask2\", np.array([False,True,True,False,False,True]))\nworkspace.FeedBlob(\"data2\", np.array([2,3,6]))\n\nprint(\"data1:\", workspace.FetchBlob(\"data1\"))\nprint(\"mask1:\", workspace.FetchBlob(\"mask1\"))\nprint(\"data2:\", workspace.FetchBlob(\"data2\"))\nprint(\"mask2:\", workspace.FetchBlob(\"mask2\"))\nworkspace.RunOperatorOnce(op)\nprint(\"unmasked_data:\", workspace.FetchBlob(\"unmasked_data\"))\n\n```\n\n**Result**\n\n```\n\ndata1: [1 4 5]\nmask1: [ True False False  True  True False]\ndata2: [2 3 6]\nmask2: [False  True  True False False  True]\nunmasked_data: [1 2 3 4 5 6]\n\n```\n\n</details>\n",
      "inputs": [
        {
          "description": "(*Tensor*): 1D input tensor(s)",
          "name": "data"
        },
        {
          "description": "(*Tensor`<bool>`*): 1D boolean mask tensor(s)",
          "name": "mask"
        }
      ],
      "outputs": [
        {
          "description": "(*Tensor*): 1D tensor of same type as `data` input that contains the unmasked input tensor",
          "name": "unmasked_data"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Fused8BitRowwiseQuantizedToHalfFloat",
    "schema": {
      "description": "\nDe-quantizes the result of the\nHalfFloatToFused8BitRowwiseQuantized operator. The input is expected to\nencode the scale as a 32-bit float in the second to the last 4 bytes of each\nrow, followed by the bias as a 32-bit float in the next 4 bytes, and the\nquantized values in the preceding bytes of the row. The output is a\nmatrix containing only the values, but de-quantized. De-quantization is\nperformed by multiplying each value by its row's scale and bias\nparameters. The de-quantized values will thus not be exactly equal to\nthe original, un-quantized floating point values.\n",
      "inputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "scale_bias_quantized_input"
        }
      ],
      "outputs": [
        {
          "description": "Float16 data",
          "name": "float16_output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "LengthsGather",
    "schema": {
      "description": "\nGather items from sparse tensor. Sparse tensor is described by items and\nlengths. This operator gathers items corresponding to lengths at the given\nindices. This deliberately doesn't return lengths of OUTPUTS so that both lists\nand maps can be supported without special cases. If you need lengths tensor for\n OUTPUT, use `Gather`.\n\nExample:\n  ITEMS = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n  LENGTHS = [0, 2, 3, 1, 4]\n  INDICES = [0, 2, 4]\n\n  OUTPUT = [2, 3, 4, 6, 7, 8, 9]\n",
      "inputs": [
        {
          "description": "items tensor",
          "name": "ITEMS"
        },
        {
          "description": "lengths tensor",
          "name": "LENGTHS"
        },
        {
          "description": "indices into LENGTHS where items should be gathered",
          "name": "INDICES"
        }
      ],
      "outputs": [
        {
          "description": "1-D tensor containing gathered items",
          "name": "OUTPUT"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SequenceMask",
    "schema": {
      "attributes": [
        {
          "description": "(string) Mode selection. Possible values: 'sequence', 'upper', 'lower', 'upperdiag', 'lowerdiag'",
          "name": "mode",
          "option": "optional"
        },
        {
          "description": "(int) Beginning axis of row elements. All dimensions to the left will be treated as row indices and those to the right (inclusive) will be treated as column indices in the 2D mask",
          "name": "axis",
          "option": "optional"
        },
        {
          "description": "(bool) operate in gradient mode",
          "name": "grad",
          "option": "optional"
        },
        {
          "description": "(int) radius of windows in window mode",
          "name": "radius",
          "option": "optional"
        },
        {
          "description": "(int) batch dimension of tensor (optional)",
          "name": "batch",
          "option": "optional"
        },
        {
          "description": "(int) used when mask should be repeated for one or more data dimensions (beginning at this axis).  (currently only supported for sequence mode without batch argument)",
          "name": "repeat_from_axis",
          "option": "optional"
        }
      ],
      "description": "\nMask op designed for use in attention mechanisms for sequence modeling tasks.\nSupports batching: given batch_dim, collapses dims 0 through batch_dim into a\nsingle dimension, e.g. if tensor dims are [4,2,1,3,4] and batch_dim=2, first\ncollapse tensor to [4*2*1,3,4], then mask each batch [i,:,:].\n\n\nTwo current operating modes:\n\n\n1) Given a 2D input tensor and 1D tensor of sequence lengths, for each row i in\nthe input tensor, set elements in that row to -inf if their column index\nj >= sequence_lengths[i]. This mode takes two inputs and argument mode =\n'sequence'\n\n\n2) Triangular mask. Given row index i and column index j, set elements to -inf\ngiven the following conditions:\n\n      mode='upper', x_ij = -inf if j < i\n      mode='lower', x_ij = -inf if j > i\n      mode='upperdiag', x_ij = -inf if j <= i\n      mode='lowerdiag', x_ij = -inf if j >= i\n\nThis mode takes one input.\n\n\n3) Window Mask. Given a 2D input tensor and 1D tensor of window centers,\nfor each row i in the input tensor, set elements in that row to -inf\nif their column index j outside [center - radius, center + radius].\nThis mode takes two inputs and argument mode = 'sequence'.\nArgument 'radius' should be provided.\n",
      "inputs": [
        {
          "description": "Tensor to apply masking to",
          "name": "input"
        },
        {
          "description": "1D Tensor of sequence lengths for mode #1",
          "name": "sequence_lengths"
        }
      ],
      "outputs": [
        {
          "description": "Input tensor with masking applied",
          "name": "masked_tensor"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Int8Transpose",
    "schema": {
      "attributes": [
        {
          "description": "Order to permute axes of input tensor. Reverses the dimensions by default.",
          "name": "axes",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization scale",
          "name": "Y_scale",
          "option": "optional"
        },
        {
          "description": "Output tensor quantization offset",
          "name": "Y_zero_point",
          "option": "optional"
        }
      ],
      "description": "\nTranspose the input tensor by permuting the axes of the input according\nto the `axes` argument. Similar to numpy's\n[transpose](https://docs.scipy.org/doc/numpy/reference/generated/numpy.transpose.html)\nfunction.\n\nFor example, when axes=(1, 0, 2), given an input tensor of shape\n(1, 2, 3), the output shape will be (2, 1, 3).\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Transposed output",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "ResizeNearest3DGradient",
    "schema": {
      "attributes": [
        {
          "description": "Scale along temporal dimension",
          "name": "temporal_scale",
          "option": "optional"
        },
        {
          "description": "Scale along width dimension",
          "name": "width_scale",
          "option": "optional"
        },
        {
          "description": "Scale along height dimension",
          "name": "height_scale",
          "option": "optional"
        }
      ],
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "ResizeNearest3D",
    "schema": {
      "attributes": [
        {
          "description": "Scale along temporal dimension",
          "name": "temporal_scale",
          "option": "optional"
        },
        {
          "description": "Scale along width dimension",
          "name": "width_scale",
          "option": "optional"
        },
        {
          "description": "Scale along height dimension",
          "name": "height_scale",
          "option": "optional"
        }
      ],
      "description": "\nResizes the spatial dimensions of the input tensor using nearest neighbor\ninterpolation. The `width_scale` and `height_scale` arguments\ncontrol the size of the output, which is given by:\noutput_width = floor(input_width * width_scale)\noutput_height = floor(output_height * height_scale)\nAssumptions:\n  - Only resize height and width\n  - Both width_scale and height_scale scale are 2\n",
      "inputs": [
        {
          "description": "Input tensor",
          "name": "X"
        }
      ],
      "outputs": [
        {
          "description": "Output tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BatchPermutation",
    "schema": {
      "description": "\nBatch permutation of an input tensor X given input indices. First dimension of\nX equals batch size N. The indices stores a be permutation of N.\nThe output Y is a tensor of same shape as X, with data re-ordered according to\nthe indices within the batch size.\n\nExample of batch permutation on a 2-D tensor with batch size 4:\n  X = [\n    [1, 5, 2, 3, 4, 6, 0],\n    [4, 3, 3, 5, 2, 3, 1],\n    [2, 2, 3, 6, 0, 0, 1],\n    [0, 0, 1, 1, 2, 2, 3]\n  ]\n  indices = [2, 0, 1, 3]\n  Y = [\n    [2, 2, 3, 6, 0, 0, 1],\n    [1, 5, 2, 3, 4, 6, 0],\n    [4, 3, 3, 5, 2, 3, 1],\n    [0, 0, 1, 1, 2, 2, 3]\n  ]\n\nExample of batch permutation on a 3-D tensor with batch size 4:\n  X = [\n    [[1, 5, 2], [3, 4, 6, 0]],\n    [[4, 3, 3], [5, 2, 3, 1]],\n    [[2, 2, 3], [6, 0, 0, 1]],\n    [[0, 0, 1], [1, 2, 2, 3]]\n  ]\n  indices = [2, 0, 1, 3]\n  Y = [\n    [[2, 2, 3], [6, 0, 0, 1]],\n    [[1, 5, 2], [3, 4, 6, 0]],\n    [[4, 3, 3], [5, 2, 3, 1]],\n    [[0, 0, 1], [1, 2, 2, 3]]\n  ]\n",
      "inputs": [
        {
          "description": "Input tensor, where 1st dimension equals batch size",
          "name": "X"
        },
        {
          "description": "Input indices of batch to permute",
          "name": "indices"
        }
      ],
      "outputs": [
        {
          "description": "Output permuted tensor",
          "name": "Y"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "BatchPermutationGradient",
    "schema": {
      "description": null,
      "support_level": "default"
    }
  },
  {
    "name": "AliasWithName",
    "schema": {
      "attributes": [
        {
          "description": "name of the aliasing",
          "name": "name",
          "option": "optional"
        },
        {
          "description": "weather or not to alias forward or backward",
          "name": "is_backward",
          "option": "optional"
        }
      ],
      "description": "\nSimilar with AliasOp, storing the alias name as operator argument.\n",
      "inputs": [
        {
          "description": "Input tensor whose storage will be shared.",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Tensor of same shape as input, sharing its storage.",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RowWiseCounter",
    "schema": {
      "attributes": [
        {
          "description": "Default -1: off",
          "name": "counter_halflife",
          "option": "optional"
        }
      ],
      "description": "\n    Count the number recent update on rows. Exponential decay is\n    applied on the counter with decay rate r, such that\n    r^{counter_halflife} = 0.5; If counter_halflife is nonpositive,\n    this operator is turned off.\n",
      "inputs": [
        {
          "description": "Iter at last update",
          "name": "prev_iter"
        },
        {
          "description": "update counter",
          "name": "update_counter"
        },
        {
          "description": "Sparse indices",
          "name": "indices"
        },
        {
          "description": "current iteration",
          "name": "iter"
        }
      ],
      "outputs": [
        {
          "description": "Updated iter at last update",
          "name": "output_prev_iter"
        },
        {
          "description": "Output update counter",
          "name": "output_update_counter"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Fused8BitRowwiseQuantizedHalfScaleBiasToHalfFloat",
    "schema": {
      "description": "\nDe-quantizes the result of the\nFloatToFused8BitRowwiseQuantized operator. The input is expected to\nencode the scale as a 16-bit float in the second to the last 2 bytes of each\nrow, followed by the bias as a 16-bit float in the next 2 bytes, and the\nquantized values in the preceding bytes of the row. The output is a\nmatrix containing only the values, but de-quantized. De-quantization is\nperformed by multiplying each value by its row's scale and bias\nparameters. The de-quantized values will thus not be exactly equal to\nthe original, un-quantized floating point values.\n",
      "inputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "scale_bias_quantized_input"
        }
      ],
      "outputs": [
        {
          "description": "Float32 data",
          "name": "float_output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FloatToFused8BitRowwiseQuantizedHalfScaleBias",
    "schema": {
      "description": "\nApplies 8-bit row-wise quantization by determining the range\n(maximum - minimum) and offset (minimum value) of each row in the input\nmatrix, and then scaling each element to an 8-bit number between 0 and\n255. To later de-quantize values, the scale (range / 255) and offset\n(bias) are stored alongside the data. More precisely, each row contains\nint8 elements for each quantized element, and the last 4 bytes\nof each row in the output matrix are a half float storing the scale\nfollowed by another half float containing the scale.)\n",
      "inputs": [
        {
          "description": "Float32 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HalfFloatToFused8BitRowwiseQuantizedHalfScaleBias",
    "schema": {
      "description": "\nApplies 8-bit row-wise quantization by determining the range\n(maximum - minimum) and offset (minimum value) of each row in the input\nmatrix, and then scaling each element to an 8-bit number between 0 and\n255. To later de-quantize values, the scale (range / 255) and offset\n(bias) are stored alongside the data. More precisely, each row contains\nint8 elements for each quantized element, and the last 4 bytes\nof each row in the output matrix are a float storing the scale\nfollowed by another float containing the scale.)\n",
      "inputs": [
        {
          "description": "Float16 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Fused8BitRowwiseQuantizedHalfScaleBiasToFloat",
    "schema": {
      "description": "\nDe-quantizes the result of the\nFloatToFused8BitRowwiseQuantized operator. The input is expected to\nencode the scale as a 16-bit float in the second to the last 2 bytes of each\nrow, followed by the bias as a 16-bit float in the next 2 bytes, and the\nquantized values in the preceding bytes of the row. The output is a\nmatrix containing only the values, but de-quantized. De-quantization is\nperformed by multiplying each value by its row's scale and bias\nparameters. The de-quantized values will thus not be exactly equal to\nthe original, un-quantized floating point values.\n",
      "inputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "scale_bias_quantized_input"
        }
      ],
      "outputs": [
        {
          "description": "Float32 data",
          "name": "float_output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "AtomicFetchAdd64",
    "schema": {
      "description": "\nLike, AtomicFetchAdd but with int64_t scalar tensors,\nperforms an atomic fetch add\nby mutating the first argument and adding it to the second input\nargument. Returns the updated integer and the value prior to the update.\n",
      "inputs": [
        {
          "description": "Blob containing to a unique_ptr<mutex>",
          "name": "mutex_ptr"
        },
        {
          "description": "Value to be mutated after the sum.",
          "name": "mut_value"
        },
        {
          "description": "Value to add to the first operand.",
          "name": "increment"
        }
      ],
      "outputs": [
        {
          "description": "Mutated value after sum. Usually same as input 1.",
          "name": "mut_value"
        },
        {
          "description": "Value of the first operand before sum.",
          "name": "fetched_value"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Quantile",
    "schema": {
      "attributes": [
        {
          "description": "If true (default), apply abs() on the tensor values.",
          "name": "abs",
          "option": "optional"
        },
        {
          "description": "multiplicative tolerance of the quantile_value.",
          "name": "tol",
          "option": "optional"
        }
      ],
      "description": "\n    Calculate the quantile for the value in the given list of tensors.\n",
      "inputs": [
        {
          "description": "*(type: Tensor`<float>`)* List of input tensors.",
          "name": "X1, X2, ..."
        }
      ],
      "outputs": [
        {
          "description": "Value at the given quantile",
          "name": "quantile_value"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsSumFused2BitRowwise",
    "schema": {
      "description": "\nPerforms the same operation as SparseLengthsSum, but operating on\n2-bit rowwise quantized matrices with fused storage (where each row\nstores quantized values, and then 2-byte fp16 scale and bias).\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused2BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FloatToFused4BitRowwiseQuantized",
    "schema": {
      "description": "\nApplies 4-bit row-wise quantization by determining the range\n(maximum - minimum) and offset (minimum value) of each row in the input\nmatrix, and then scaling each element to an 4-bit number between 0 and\n15. To later de-quantize values, the scale (range / 15) and zero_point\nare stored alongside the data. More precisely, each row first has quantized\nvalues, and then 2-byte fp16 scale and 2-byte zero_offset.)\n",
      "inputs": [
        {
          "description": "Float32 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsWeightedSum2BitRowwiseSparse",
    "schema": {
      "description": "\nPerforms SparseLengthsWeightedSum, but operating on 2-bit rowwise quantized\nmatrices with fused storage (where each row stores quantized values, and then\n2-byte fp16 scale and bias), and where rows are pruned.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused2BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Vector of weights to scale rows of DATA with before reduction",
          "name": "WEIGHTS"
        },
        {
          "description": "Integer vector mapping uncompressed indices to compressed indices",
          "name": "COMPRESSED_INDICES_MAPPING"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FloatToFused2BitRowwiseQuantized",
    "schema": {
      "description": "\nApplies 2-bit row-wise quantization by determining the range\n(maximum - minimum) and offset (minimum value) of each row in the input\nmatrix, and then scaling each element to an 2-bit number between 0 and\n3. To later de-quantize values, the scale (range / 3) and zero_point\nare stored alongside the data. More precisely, each row first has quantized\nvalues, and then 2-byte fp16 scale and 2-byte zero_offset.)\n",
      "inputs": [
        {
          "description": "Float32 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsWeightedSum4BitRowwiseSparse",
    "schema": {
      "description": "\nPerforms SparseLengthsWeightedSum, but operating on 4-bit rowwise quantized\nmatrices with fused storage (where each row stores quantized values, and then\n2-byte fp16 scale and bias), and where rows are pruned.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused4BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Vector of weights to scale rows of DATA with before reduction",
          "name": "WEIGHTS"
        },
        {
          "description": "Integer vector mapping uncompressed indices to compressed indices",
          "name": "COMPRESSED_INDICES_MAPPING"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsMean4BitRowwiseSparse",
    "schema": {
      "description": "\nPerforms SparseLengthsMean, but operating on 4-bit rowwise quantized matrices\nwith fused storage (where each row stores quantized values, and then 2-byte\nfp16 scale and bias), and where rows are pruned.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused4BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Integer vector mapping uncompressed indices to compressed indices",
          "name": "COMPRESSED_INDICES_MAPPING"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsMean2BitRowwiseSparse",
    "schema": {
      "description": "\nPerforms SparseLengthsMean, but operating on 2-bit rowwise quantized matrices\nwith fused storage (where each row stores quantized values, and then 2-byte\nfp16 scale and bias), and where rows are pruned.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused2BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Integer vector mapping uncompressed indices to compressed indices",
          "name": "COMPRESSED_INDICES_MAPPING"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsWeightedSumFused2BitRowwise",
    "schema": {
      "description": "\nPerforms the same operation as SparseLengthsWeightedSum,\nbut operating on 2-bit rowwise quantized matrices with fused storage\n(where each row stores quantized values, and then 2-byte fp16 scale and bias).\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused2BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Vector of weights to scale rows of DATA with before reduction",
          "name": "WEIGHTS"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Fused4BitRowwiseQuantizedToFloat",
    "schema": {
      "description": "\nDe-quantizes the result of the\nFloatToFused4BitRowwiseQuantized operator. The input is expected to first have\nquantized values, then 2-byte fp16 scale and 1-byte zero_offset. The output is a\nmatrix containing only the values, but de-quantized. De-quantization is\nperformed by multiplying each value by its row's scale and zero_point\nparameters. The de-quantized values will thus not be exactly equal to\nthe original, un-quantized floating point values.\n",
      "inputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "scale_bias_quantized_input"
        }
      ],
      "outputs": [
        {
          "description": "Float32 data",
          "name": "float_output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsMean8BitRowwiseSparse",
    "schema": {
      "description": "\nPerforms SparseLengthsMean, but operating on 8-bit rowwise quantized matrices\nwith fused storage (where each row stores quantized values, and then 4-byte\nfp32 scale and bias), and where rows are pruned.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused4BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Integer vector mapping uncompressed indices to compressed indices",
          "name": "COMPRESSED_INDICES_MAPPING"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HalfToFused4BitRowwiseQuantized",
    "schema": {
      "description": "\nApplies 4-bit row-wise quantization by determining the range\n(maximum - minimum) and offset (minimum value) of each row in the input\nmatrix, and then scaling each element to an 4-bit number between 0 and\n15. To later de-quantize values, the scale (range / 15) and zero_point\nare stored alongside the data. More precisely, each row first has quantized\nvalues, and then 2-byte fp16 scale and 2-byte zero_offset.)\n",
      "inputs": [
        {
          "description": "Float16 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsSumFused4BitRowwise",
    "schema": {
      "description": "\nPerforms the same operation as SparseLengthsSum, but operating on\n4-bit rowwise quantized matrices with fused storage (where each row\nstores quantized values, and then 2-byte fp16 scale and bias).\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused4BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Fused4BitRowwiseQuantizedToHalf",
    "schema": {
      "description": "\nDe-quantizes the result of the\nFloatToFused4BitRowwiseQuantized operator. The input is expected to first have\nquantized values, then 2-byte fp16 scale and 1-byte zero_offset. The output is a\nmatrix containing only the values, but de-quantized. De-quantization is\nperformed by multiplying each value by its row's scale and zero_point\nparameters. The de-quantized values will thus not be exactly equal to\nthe original, un-quantized floating point values.\n",
      "inputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "scale_bias_quantized_input"
        }
      ],
      "outputs": [
        {
          "description": "Float16 data",
          "name": "float16_output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsMeanFused4BitRowwise",
    "schema": {
      "description": "\nPerforms the same operation as SparseLengthsMean, but\noperating on 4-bit rowwise quantized matrices with fused storage\n(where each row stores quantized values, and then 2-byte fp16 scale and bias).\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused4BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HalfToFused2BitFakeRowwiseQuantized",
    "schema": {
      "description": "\nApplies 2-bit row-wise fake quantization to a tensor of half floats.\nThe output looks like an int8 rowwise quantized blob with\nscale and biases in half float.\n",
      "inputs": [
        {
          "description": "Float16 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FloatToFused2BitFakeRowwiseQuantized",
    "schema": {
      "description": "\nApplies 2-bit row-wise fake quantization to a tensor of floats.\nThe output looks like an int8 rowwise quantized blob with\nscale and biases in half float.\n",
      "inputs": [
        {
          "description": "Float32 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "FloatToFused4BitFakeRowwiseQuantized",
    "schema": {
      "description": "\nApplies 4-bit row-wise fake quantization to a tensor of floats.\nThe output looks like an int8 rowwise quantized blob with\nscale and biases in half float.\n",
      "inputs": [
        {
          "description": "Float32 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Fused2BitRowwiseQuantizedToFloat",
    "schema": {
      "description": "\nDe-quantizes the result of the\nFloatToFused2BitRowwiseQuantized operator. The input is expected to first have\nquantized values, then 2-byte fp16 scale and 1-byte zero_offset. The output is a\nmatrix containing only the values, but de-quantized. De-quantization is\nperformed by multiplying each value by its row's scale and zero_point\nparameters. The de-quantized values will thus not be exactly equal to\nthe original, un-quantized floating point values.\n",
      "inputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "scale_bias_quantized_input"
        }
      ],
      "outputs": [
        {
          "description": "Float32 data",
          "name": "float_output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsSum4BitRowwiseSparse",
    "schema": {
      "description": "\nPerforms SparseLengthsSum, but operating on 4-bit rowwise quantized matrices\nwith fused storage (where each row stores quantized values, and then 2-byte\nfp16 scale and 2-byte fp16 bias), and where rows are pruned.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused4BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Integer vector mapping uncompressed indices to compressed indices",
          "name": "COMPRESSED_INDICES_MAPPING"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HalfToFused2BitRowwiseQuantized",
    "schema": {
      "description": "\nApplies 2-bit row-wise quantization by determining the range\n(maximum - minimum) and offset (minimum value) of each row in the input\nmatrix, and then scaling each element to an 2-bit number between 0 and\n3. To later de-quantize values, the scale (range / 3) and zero_point\nare stored alongside the data. More precisely, each row first has quantized\nvalues, and then 2-byte fp16 scale and 2-byte zero_offset.)\n",
      "inputs": [
        {
          "description": "Float16 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsMeanFused2BitRowwise",
    "schema": {
      "description": "\nPerforms the same operation as SparseLengthsMean, but\noperating on 2-bit rowwise quantized matrices with fused storage\n(where each row stores quantized values, and then 2-byte fp16 scale and bias).\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused2BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsWeightedSumFused4BitRowwise",
    "schema": {
      "description": "\nPerforms the same operation as SparseLengthsWeightedSum,\nbut operating on 4-bit rowwise quantized matrices with fused storage\n(where each row stores quantized values, and then 2-byte fp16 scale and bias).\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused4BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Vector of weights to scale rows of DATA with before reduction",
          "name": "WEIGHTS"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsSum8BitRowwiseSparse",
    "schema": {
      "description": "\nPerforms SparseLengthsSum, but operating on 8-bit rowwise quantized matrices\nwith fused storage (where each row stores quantized values, and then 4-byte\nfp32 scale and 4-byte fp32 bias), and where rows are pruned.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused4BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Integer vector mapping uncompressed indices to compressed indices",
          "name": "COMPRESSED_INDICES_MAPPING"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsSum2BitRowwiseSparse",
    "schema": {
      "description": "\nPerforms SparseLengthsSum, but operating on 2-bit rowwise quantized matrices\nwith fused storage (where each row stores quantized values, and then 2-byte\nfp16 scale and 2-byte fp16 bias), and where rows are pruned.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused2BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Integer vector mapping uncompressed indices to compressed indices",
          "name": "COMPRESSED_INDICES_MAPPING"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "HalfToFused4BitFakeRowwiseQuantized",
    "schema": {
      "description": "\nApplies 4-bit row-wise fake quantization to a tensor of half floats.\nThe output looks like an int8 rowwise quantized blob with\nscale and biases in half float.\n",
      "inputs": [
        {
          "description": "Float16 input data",
          "name": "input"
        }
      ],
      "outputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsWeightedSum8BitRowwiseSparse",
    "schema": {
      "description": "\nPerforms SparseLengthsWeightedSum, but operating on 8-bit rowwise quantized\nmatrices with fused storage (where each row stores quantized values, and then\n4-byte fp32 scale and bias), and where rows are pruned.\n",
      "inputs": [
        {
          "description": "uint8 tensor obtained with operator FloatToFused4BitRowwiseQuantized",
          "name": "DATA"
        },
        {
          "description": "Integer vector containing indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of DATA",
          "name": "LENGTHS"
        },
        {
          "description": "Vector of weights to scale rows of DATA with before reduction",
          "name": "WEIGHTS"
        },
        {
          "description": "Integer vector mapping uncompressed indices to compressed indices",
          "name": "COMPRESSED_INDICES_MAPPING"
        }
      ],
      "outputs": [
        {
          "description": "output",
          "name": "output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "Fused2BitRowwiseQuantizedToHalf",
    "schema": {
      "description": "\nDe-quantizes the result of the\nFloatToFused2BitRowwiseQuantized operator. The input is expected to first have\nquantized values, then 2-byte fp16 scale and 1-byte zero_offset. The output is a\nmatrix containing only the values, but de-quantized. De-quantization is\nperformed by multiplying each value by its row's scale and zero_point\nparameters. The de-quantized values will thus not be exactly equal to\nthe original, un-quantized floating point values.\n",
      "inputs": [
        {
          "description": "Fused scale, bias and quantized data",
          "name": "scale_bias_quantized_input"
        }
      ],
      "outputs": [
        {
          "description": "Float16 data",
          "name": "float16_output"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "WeightScale",
    "schema": {
      "attributes": [
        {
          "description": "Every iteration number to do weight scaling",
          "name": "stepsize",
          "option": "optional"
        },
        {
          "description": "After iter passes this bound, do not perform the weight rescaling",
          "name": "upper_bound_iter",
          "option": "optional"
        },
        {
          "description": "The multiplicative factor applied to weights.",
          "name": "scale",
          "option": "optional"
        }
      ],
      "description": "\nEvery `stepsize` iterations, multiply the weights by a constant `scale`:\n    nw = w * scale\n",
      "inputs": [
        {
          "description": "Current weights",
          "name": "w"
        },
        {
          "description": "Training Iteration",
          "name": "iter"
        }
      ],
      "outputs": [
        {
          "description": "Updated weights",
          "name": "nw"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseAdagradFusedWithSparseLengthsSumGradient",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\nFused operator of\nSparseLengthsIndicesInGradientSumGradient (gradient of SparseLengthsSum) +\nSparseAdagrad.\n\nGiven inputs (param, moment, indices, grad, lr), runs the sparse AdaGrad\nupdate on (param, grad, moment[indices], lr), and returns (new_param,\nnew_moment) as in the dense case. Additional input (lengths) is for fused\nSparseLengthsIndicesInGradientSumGradient operator.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Moment history",
          "name": "moment"
        },
        {
          "description": "Integer vector containing indices of the first dimension of param for the slices that are being updated",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        },
        {
          "description": "Non negative vector with sum of elements equal to indices length",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated moment",
          "name": "output_moment"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseAdagradFusedWithSparseLengthsWeightedSumGradientApprox",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\nApproximately fused operator of\nSparseLengthsIndicesInGradientWeightedSumWithMainInputGradient\n(gradient of SparseLengthsWeightedSum) + SparseAdagrad, where weights are\npositional weights computed with LengthsRangeFill + Gather pattern.\n\nGiven inputs (param, moment, indices, grad, lr), runs the sparse AdaGrad\nupdate on (param, grad, moment[indices], lr), and returns (new_param,\nnew_moment) as in the dense case.\nThere's race condition w.r.t. ordering between reading params and writing to\nparam, hence the name Approx.\nThere're auxiliary inputs (aux_param) for which gradient is computed and\nreturns (aux_grad).\nYet additional input (lengths) is for fused\nSparseLengthsIndicesInGradientWeightedSumWithMainInputGradient operator.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Moment history",
          "name": "moment"
        },
        {
          "description": "Auxiliary parameters to be updated",
          "name": "aux_param"
        },
        {
          "description": "Integer vector containing indices of the first dimension of param for the slices that are being updated",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        },
        {
          "description": "Non negative vector with sum of elements equal to indices length",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated moment",
          "name": "output_moment"
        },
        {
          "description": "Auxiliary gradients",
          "name": "aux_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RowWiseSparseAdagradFusedWithSparseLengthsWeightedSumGradient",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\nFused operator of SparseLengthsIndicesInGradientWeightedSumWithMainInputGradient\n(gradient of SparseLengthsWeightedSum) + RowWiseSparseAdagrad, where weights are\npositional weights computed with LengthsRangeFill + Gather pattern.\n\nGiven inputs (param, moment, indices, grad, lr), runs the row-wise sparse\nAdaGrad update on (param, grad, moment[indices], lr), and returns (new_param,\nnew_moment) as in the dense case.\nThere're auxiliary inputs (aux_param) for which gradient is computed and\nreturns (aux_grad).\nYet additional input (lengths) is for fused SparseLengthsWeightedSumGradient\noperator.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Moment history",
          "name": "moment"
        },
        {
          "description": "Auxiliary parameters to be updated",
          "name": "aux_param"
        },
        {
          "description": "Integer vector containing indices of the first dimension of param for the slices that are being updated",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        },
        {
          "description": "Non negative vector with sum of elements equal to indices length",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated moment",
          "name": "output_moment"
        },
        {
          "description": "Auxiliary gradient",
          "name": "aux_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RowWiseSparseAdagradFusedWithSparseLengthsSumGradient",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\nFused operator of\nSparseLengthsIndicesInGradientSumGradient (gradient of SparseLengthsSum) +\nRowWiseSparseAdagrad.\n\nGiven inputs (param, moment, indices, grad, lr), runs the row-wise sparse\nAdaGrad update on (param, grad, moment[indices], lr), and returns (new_param,\nnew_moment) as in the dense case. Additional input (lengths) is for fused\nSparseLengthsSumGradient operator.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Moment history",
          "name": "moment"
        },
        {
          "description": "Integer vector containing indices of the first dimension of param for the slices that are being updated",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        },
        {
          "description": "Non negative vector with sum of elements equal to indices length",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated moment",
          "name": "output_moment"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseAdagradFusedWithSparseLengthsWeightedSumGradient",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\nFused operator of SparseLengthsIndicesInGradientWeightedSumWithMainInputGradient\n(gradient of SparseLengthsWeightedSum) + SparseAdagrad, where weights are\npositional weights computed with LengthsRangeFill + Gather pattern.\n\nGiven inputs (param, moment, indices, grad, lr), runs the sparse AdaGrad\nupdate on (param, grad, moment[indices], lr), and returns (new_param,\nnew_moment) as in the dense case.\nThere're auxiliary inputs (aux_param) for which gradient is computed\nand returns (aux_grad).\nYet additional input (lengths) is for fused\nSparseLengthsIndicesInGradientWeightedSumWithMainInputGradient operator.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Moment history",
          "name": "moment"
        },
        {
          "description": "Auxiliary parameters to be updated",
          "name": "aux_param"
        },
        {
          "description": "Integer vector containing indices of the first dimension of param for the slices that are being updated",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        },
        {
          "description": "Non negative vector with sum of elements equal to indices length",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated moment",
          "name": "output_moment"
        },
        {
          "description": "Auxiliary gradient",
          "name": "aux_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "RowWiseSparseAdagradFusedWithSparseLengthsWeightedSumGradientApprox",
    "schema": {
      "attributes": [
        {
          "description": "Default 1e-5",
          "name": "epsilon",
          "option": "optional"
        }
      ],
      "description": "\n\nApproximately fused operator of\nSparseLengthsIndicesInGradientWeightedSumWithMainInputGradient\n(gradient of SparseLengthsWeightedSum) + RowWiseSparseAdagrad, where weights are\npositional weights computed with LengthsRangeFill + Gather pattern.\n\nGiven inputs (param, moment, indices, grad, lr), runs the row-wise sparse\nAdaGrad update on (param, grad, moment[indices], lr), and returns (new_param,\nnew_moment) as in the dense case.\nThere's race condition w.r.t. ordering between reading params and writing to\nparam, hence the name Approx.\nThere're auxiliary inputs (aux_param) for which gradient is computed\nand returns (aux_grad).\nYet additional input (lengths) is for fused SparseLengthsWeightedSumGradient\noperator.\n\n",
      "inputs": [
        {
          "description": "Parameters to be updated",
          "name": "param"
        },
        {
          "description": "Moment history",
          "name": "moment"
        },
        {
          "description": "Auxiliary parameters to be updated",
          "name": "aux_param"
        },
        {
          "description": "Integer vector containing indices of the first dimension of param for the slices that are being updated",
          "name": "indices"
        },
        {
          "description": "Gradient computed",
          "name": "grad"
        },
        {
          "description": "learning rate",
          "name": "lr"
        },
        {
          "description": "Non negative vector with sum of elements equal to indices length",
          "name": "lengths"
        }
      ],
      "outputs": [
        {
          "description": "Updated parameters",
          "name": "output_param"
        },
        {
          "description": "Updated moment",
          "name": "output_moment"
        },
        {
          "description": "Auxiliary gradient",
          "name": "aux_grad"
        }
      ],
      "support_level": "default"
    }
  },
  {
    "name": "SparseLengthsSumSparseLookup",
    "schema": {
      "description": "\nThis op converts compressed indices of SparseLengthsSum*Sparse to\nuncompressed indices of SparseLengthsSum*. For compressed indices that maps\nto -1. It means it will correspond to a zero row in the uncompressed data.\nTherefore we will remove this indices and adjust the lengths.\n",
      "inputs": [
        {
          "description": "Integer vector containing compressed indices of the first dimension of DATA for the slices that are being aggregated",
          "name": "INDICES"
        },
        {
          "description": "Vector with the same sum of elements as the first dimension of INDICES",
          "name": "LENGTHS"
        },
        {
          "description": "Integer vector mapping uncompressed indices to compressed indices",
          "name": "COMPRESSED_INDICES_MAPPING"
        },
        {
          "description": "Vector of weights to scale rows of DATA with before reduction. Same size as INDICES.",
          "name": "WEIGHTS"
        }
      ],
      "outputs": [
        {
          "description": "Uncompressed indices",
          "name": "output_indices"
        },
        {
          "description": "Adjusted lengths",
          "name": "output_lengths"
        },
        {
          "description": "Adjusted weights",
          "name": "output_weights"
        }
      ],
      "support_level": "default"
    }
  }
]
