PyTorch: index 2D tensor with 2D tensor of row indices - python

I have a torch tensor a of shape (x, n) and another tensor b of shape (y, n) where y <= x. every column of b contains a sequence of row indices for a and what I would like to be able to do is to somehow index a with b such that I obtain a tensor of shape (y, n) in which the ith column contains a[:, i][b[:, i]] (not quite sure if that's the correct way to express it).
Here's an example (where x = 5, y = 3 and n = 4):
import torch
a = torch.Tensor(
[[0.1, 0.2, 0.3, 0.4],
[0.6, 0.7, 0.8, 0.9],
[1.1, 1.2, 1.3, 1.4],
[1.6, 1.7, 1.8, 1.9],
[2.1, 2.2, 2.3, 2.4]]
)
b = torch.LongTensor(
[[0, 3, 1, 2],
[2, 2, 2, 0],
[1, 1, 0, 4]]
)
# How do I get from a and b to c
# (so that I can also assign to those elements in a)?
c = torch.Tensor(
[[0.1, 1.7, 0.8, 1.4],
[1.1, 1.2, 1.3, 0.4],
[0.6, 0.7, 0.3, 2.4]]
)
I can't get my head around this. What I'm looking for is a method that will not yield the tensor c but also let me assign a tensor of the same shape as c to the elements of a which c is made up of.

I try to use index_select but it supports only 1-dim array for index.
bt = b.transpose(0, 1)
at = a.transpose(0, 1)
ct = [torch.index_select(at[i], dim=0, index=bt[i]) for i in range(len(at))]
c = torch.stack(ct).transpose(0, 1)
print(c)
"""
tensor([[0.1000, 1.7000, 0.8000, 1.4000],
[1.1000, 1.2000, 1.3000, 0.4000],
[0.6000, 0.7000, 0.3000, 2.4000]])
"""
It might be not the best solution, but hope this helps you at least.

Related

Adding unique labels with matplotlib in for loop

Is it possible to add a different label to each line that's plotted in a for loop? I want the label to depend on the iterating element i. Say I have a list of length 4 called my_list, and I want to do the following:
import matplotlib.pyplot as plt
my_list = [1, 2, 3, 4]
other_list = [[1, 1.2, 1.4, 1.6], [0.8, 0.9, 0.7, 0.6], [1.9, 1.3, 1.4, 1.0], [0.1, 0.2, 0.4, 0.6]]
for i in range(len(my_list)):
plt.plot(my_list, other_list[i], label='Line %s' % i)
plt.legend('upper left')
I want the labels to appear as Line 0, ..., Line 3 depending on what i is in the loop. But I'm not getting this -- my labels are appearing as letters u, p, p, and e for some reason. I've also tried label='Line'+str(i), and got the same output.

How to compute the weighted sum of a tensor in TensorFlow?

I want to compute the weighted sum of every row of a tensor, the weight is the element and the other multiplier is the column number. Example blew:
input:
[[0.2 0.5 0.3],
[0.4 0.1 0.5]]
output:
[0.2*0+0.5*1+0.3*2, 0.4*0+0.1*1+0.5*2] = [1.1, 1.1]
How should I do with it?
This is the very definition of inner product. With numpy.dot:
data=[[0.2, 0.5, 0.3], [0.4, 0.1, 0.5]]
np.dot(data,range(len(data)+1))
array([ 1.1, 1.1])
The following piece of code should work:
import tensorflow as tf
import numpy as np
a = np.array([[0.2, 0.5, 0.3], [0.4, 0.1, 0.5]])
x = tf.placeholder(tf.float64, [2, 3])
# weighted_sum_op = tf.reduce_sum(x * np.arange(0, a.shape[1]), 1,)
# or if you want to have the range in the TensorFlow graph:
weighted_sum_op = tf.reduce_sum(x * tf.range(0., tf.cast(tf.shape(x)[1], tf.float64)), 1, )
# You ccould also make use of tf.py_func
# weighted_sum_op = tf.py_func(lambda y: np.dot(y, np.arange(0, y.shape[1])), [x], tf.float64)
with tf.Session() as sess:
print(sess.run(weighted_sum_op, {x: a}))

“Unsorting” a Quicksort

(Quick note! While I know there are plenty of options for sorting in Python, this code is more of a generalized proof-of-concept and will later be ported to another language, so I won't be able to use any specific Python libraries or functions.
In addition, the solution you provide doesn't necessarily have to follow my approach below.)
Background
I have a quicksort algorithm and am trying to implement a method to allow later 'unsorting' of the new location of a sorted element. That is, if element A is at index x and is sorted to index y, then the 'pointer' (or, depending on your terminology, reference or mapping) array changes its value at index x from x to y.
In more detail:
You begin the program with an array, arr, with some given set of numbers. This array is later run through a quick sort algorithm, as sorting the array is important for future processing on it.
The ordering of this array is important. As such, you have another array, ref, which contains the indices of the original array such that when you map the reference array to the array, the original ordering of the array is reproduced.
Before the array is sorted, the array and mapping looks like this:
arr = [1.2, 1.5, 1.5, 1.0, 1.1, 1.8]
ref = [0, 1, 2, 3, 4, 5]
--------
map(arr,ref) -> [1.2, 1.5, 1.5, 1.0, 1.1, 1.8]
You can see that index 0 of ref points to index 0 of arr, giving you 1.2. Index 1 of ref points to index 1 of arr, giving you 1.5, and so on.
When the algorithm is sorted, ref should be rearranged such that when you map it according to the above procedure, it generates the pre-sorted arr:
arr = [1.0, 1.1, 1.2, 1.5, 1.5, 1.8]
ref = [2, 3, 4, 0, 1, 5]
--------
map(arr,ref) -> [1.2, 1.5, 1.5, 1.0, 1.1, 1.8]
Again, index 0 of ref is 2, so the first element of the mapped array is arr[2]=1.2. Index 1 of ref is 3, so the second element of the mapped array is arr[3]=1.5, and so on.
The Issue
The current implementation of my code works great for sorting, but horrible for the remapping of ref.
Given the same array arr, the output of my program looks like this:
arr = [1.0, 1.1, 1.2, 1.5, 1.5, 1.8]
ref = [3, 4, 0, 1, 2, 5]
--------
map(arr,ref) -> [1.5, 1.5, 1.0, 1.1, 1.2, 1.8]
This is a problem because this mapping is definitely not equal to the original:
[1.5, 1.5, 1.0, 1.1, 1.2, 1.8] != [1.2, 1.5, 1.5, 1.0, 1.1, 1.8]
My approach has been this:
When elements a and b, at indices x and y in arr are switched,
Then set ref[x] = y and ref[y] = x.
This is not working and I can't think of another solution that doesn't need O(n^2) time.
Thank you!
Minimally Reproducible Example
testing = [1.5, 1.2, 1.0, 1.0, 1.2, 1.2, 1.5, 1.3, 2.0, 0.7, 0.2, 1.4, 1.2, 1.8, 2.0, 2.1]
# This is the 'map(arr,ref) ->' function
def print_links(a,b):
tt = [a[b[i]-1] for i in range(0,len(a))]
print("map(arr,ref) -> {}".format(tt))
# This tests the re-mapping against an original copy of the array
f = 0
for i in range(0,len(testing)):
if testing[i] == tt[i]:
f += 1
print("{}/{}".format(f,len(a)))
def quick_sort(arr,ref,first=None,last=None):
if first == None:
first = 0
if last == None:
last = len(arr)-1
if first < last:
split = partition(arr,ref,first,last)
quick_sort(arr,ref,first,split-1)
quick_sort(arr,ref,split+1,last)
def partition(arr,ref,first,last):
pivot = arr[first]
left = first+1
right = last
done = False
while not done:
while left <= right and arr[left] <= pivot:
left += 1
while arr[right] >= pivot and right >= left:
right -= 1
if right < left:
done = True
else:
temp = arr[left]
arr[left] = arr[right]
arr[right] = temp
# This is my attempt at preserving indices part 1
temp = ref[left]
ref[left] = ref[right]
ref[right] = temp
temp = arr[first]
arr[first] = arr[right]
arr[right] = temp
# This is my attempt at preserving indices part 2
temp = ref[first]
ref[first] = ref[right]
ref[right] = temp
return right
# Main body of code
a = [1.5,1.2,1.0,1.0,1.2,1.2,1.5,1.3,2.0,0.7,0.2,1.4,1.2,1.8,2.0,2.1]
b = range(1,len(a)+1)
print("The following should match:")
print("a = {}".format(a))
a0 = a[:]
print("ref = {}".format(b))
print("----")
print_links(a,b)
print("\nQuicksort:")
quick_sort(a,b)
print(a)
print("\nThe following should match:")
print("arr = {}".format(a0))
print("ref = {}".format(b))
print("----")
print_links(a,b)
You don't need to maintain the map of indices and elements,just sort the indices as you sort your array.for example:
unsortedArray = [1.2, 1.5, 2.1]
unsortedIndexes = [0, 1, 2]
sortedAray = [1.2, 1.5, 2.1]
then you just swap 0 and 1as you sort unsortedArray.and get the sortedIndexes[1, 0, 2],you can get the origin array by sortedArray[1],sortedArray[0],sortedArray[2].
def inplace_quick_sort(s, indexes, start, end):
if start>= end:
return
pivot = getPivot(s, start, end)#it's should be a func
left = start
right = end - 1
while left <= right:
while left <= right and customCmp(pivot, s[left]):
# s[left] < pivot:
left += 1
while left <= right and customCmp(s[right], pivot):
# pivot < s[right]:
right -= 1
if left <= right:
s[left], s[right] = s[right], s[left]
indexes[left], indexes[right] = indexes[right], indexes[left]
left, right = left + 1, right -1
s[left], s[end] = s[end], s[left]
indexes[left], indexes[end] = indexes[end], indexes[left]
inplace_quick_sort(s, indexes, start, left-1)
inplace_quick_sort(s, indexes, left+1, end)
def customCmp(a, b):
return a > b
def getPivot(s, start, end):
return s[end]
if __name__ == '__main__':
arr = [1.5,1.2,1.0,1.0,1.2,1.2,1.5,1.3,2.0,0.7,0.2,1.4,1.2,1.8,2.0,2.1]
indexes = [i for i in range(len(arr))]
inplace_quick_sort(arr,indexes, 0, len(arr)-1)
print("sorted = {}".format(arr))
ref = [0]*len(indexes)
for i in range(len(indexes)):
#the core point of Matt Timmermans' answer about how to construct the ref
#the value of indexes[i] is index of the orignal array
#and i is the index of the sorted array,
#so we get the map by ref[indexes[i]] = i
ref[indexes[i]] = i
unsorted = [arr[ref[i]] for i in range(len(ref))]
print("unsorted after sorting = {}".format(unsorted))
You can do what you ask, but when we have to do something like this in real life, we usually mess with the sort's comparison function instead of the swap function. Sorting routines provided with common languages usually have that capability built in so you don't have to write your own sort.
In this procedure, you sort the ref array (called order below), by the value of the arr value it points to. The generates the same ref array you already have, but without modifying arr.
Mapping with this ordering sorts the original array. You expected it to unsort the sorted array, which is why your code isn't working.
You can invert this ordering to get the ref array you were originally looking for, or you can just leave arr unsorted and map it through order when you need it ordered.
arr = [1.5, 1.2, 1.0, 1.0, 1.2, 1.2, 1.5, 1.3, 2.0, 0.7, 0.2, 1.4, 1.2, 1.8, 2.0, 2.1]
order = range(len(arr))
order.sort(key=lambda i:arr[i])
new_arr = [arr[order[i]] for i in range(len(arr))]
print("original array = {}".format(arr))
print("sorted ordering = {}".format(order))
print("sorted array = {}".format(new_arr))
ref = [0]*len(order)
for i in range(len(order)):
ref[order[i]]=i
unsorted = [new_arr[ref[i]] for i in range(len(ref))]
print("unsorted after sorting = {}".format(unsorted))
Output:
original array = [1.5, 1.2, 1.0, 1.0, 1.2, 1.2, 1.5, 1.3, 2.0, 0.7, 0.2, 1.4, 1.2, 1.8, 2.0, 2.1]
sorted ordering = [10, 9, 2, 3, 1, 4, 5, 12, 7, 11, 0, 6, 13, 8, 14, 15]
sorted array = [0.2, 0.7, 1.0, 1.0, 1.2, 1.2, 1.2, 1.2, 1.3, 1.4, 1.5, 1.5, 1.8, 2.0, 2.0, 2.1]
unsorted after sorting = [1.5, 1.2, 1.0, 1.0, 1.2, 1.2, 1.5, 1.3, 2.0, 0.7, 0.2, 1.4, 1.2, 1.8, 2.0, 2.1]
It's not that horrible: you've merely reversed your reference usage. Your indices, ref, tell you how to build the sorted list from the original. However, you've used it in the opposite direction: you've applied it to the sorted list, trying to reconstruct the original. You need the inverse mapping.
Is that enough to get you to solve your problem?
I think you can just repair your ref array after the fact. From your code sample, just insert the following snippet after the call toquick_sort(a,b)
c = range(1, len(b)+1)
for i in range(0, len(b)):
c[ b[i]-1 ] = i+1
The c array should now contain the correct references.
Stealing/rewording what #Prune writes: what you have in b is the forward transformation, the sorting itself. Applying it to a0 provides the sorted list (print_links(a0,b))
You just have to revert it via looking up which element went to what position:
c=[b.index(i)+1 for i in range(1,len(a)+1)]
print_links(a,c)

Tensorflow mask from one-hot encoding

I have labels that are OHE in the form of examples = tf.placeholder(tf.int32, [batch_size]) where each example is an int in the range 0:ohe_size.
My output is in the form of a softmax probability distribution with a shape [batch_size, ohe_size]
I'm trying to work out how to create a mask that will give me just the probability distribution for each example. e.g.
probs = [[0.1, 0.6, 0.3]
[0.2, 0.1, 0.7]
[0.9, 0.1, 0.0]]
examples = [2, 2, 0]
some_mask_func(probs, example) # <- Need this function
> [0.3, 0.7, 0.9]
If I understood your example correctly, you need tf.gather_nd
range = tf.range(tf.shape(examples)[0])
indices = tf.pack([range, examples], axis=1)
result = tf.gather_nd(probs, indices)

Get elements from list using index from a list of arrays

Question:
I have a list of arrays such as:
a = [array([0, 4]), array([1, 3, 2])]
From another variable X, I want to take two subsets which are chosen by the index in each array in a.
X = [0.1, 0.7, 0.9, 0.2, 0.3]
What I want to have is now:
result_1 = [0.1, 0.3]
result_2 = [0.7, 0.2, 0.9]
My solution would be to use a for loop such as:
def getresult(X, indices):
result = []
for i in indices:
result.append(X[i])
return result
This works fine:
getresult(X, a[0])
[0.1, 0.3]
My previous experience with programming suggest, that there is a much more beautiful and simple way to do so and that this is the place to ask.
Preferably someone would know a solution that does not need looping.
Background/application:
Background: Crossvalidation for parameter optimization.
I have a list containing datapoints such as
X = [0.1, 0.7, 0.9, 0.2, 0.3]
Now I want to repeatedly take z samples out of that list (which actually is much bigger than this example). Therefore I create a new variable:
indices = np.arange(0,len(X),1)
HERE: [0, 1, 2, 3, 4]
Which then i shuffle and create nfold samples:
np.random.shuffle(indices)
nfold_indices = np.array_split(indices,nfolds)
HERE with nfolds = 2: nfold_indices = [array([0, 4]), array([1, 3, 2])]
Use a list comprehension, with a default for safety, in case an index from a is not in X:
a = [array('i', [0, 4]), array('i', [1, 3, 2])]
X = [0.1, 0.7, 0.9, 0.2, 0.3]
result = [[(X[i:]+[0])[0] for i in o] for o in a]
# ^ default
# [[0.1, 0.3], [0.7, 0.2, 0.9]]
So in the event you have an arbitrary array containing an out of range index:
a = [array('i', [0, 4]), array('i', [1, 3, 20])]
# ^ out of range
result = [[(X[i:]+[0])[0] for i in o] for o in a]
# [[0.1, 0.3], [0.7, 0.2, 0]]
# ^
Your code does not break
return [X[i] for i in indices] would work.
My first attempt, which basically follows your way of doing things in a more compact way would be:
result1, result2 = [ [X[ii] for ii in array] for array in a]
If you can be bothered to convert X to a numpy array, you can do:
X = np.array([ 0.1, 0.7, 0.9, 0.2, 0.3])
result1, result2 = X[a[0]], X[a[1]]
But this has the problem that it does not generalise well for more than a few arrays within a. Even if some people will hate me for using a lambda, another compact way of doing it that generalises better is:
results = map(lambda x: X[x], a)

Resources