Following are a few examples, Game Theory (Normal-form game) | Set 3 (Game with Mixed Strategy), Game Theory (Normal-form Game) | Set 6 (Graphical Method [2 X N] Game), Game Theory (Normal-form Game) | Set 7 (Graphical Method [M X 2] Game), Combinatorial Game Theory | Set 2 (Game of Nim), Game Theory (Normal - form game) | Set 1 (Introduction), Game Theory (Normal-form Game) | Set 4 (Dominance Property-Pure Strategy), Game Theory (Normal-form Game) | Set 5 (Dominance Property-Mixed Strategy), Minimax Algorithm in Game Theory | Set 1 (Introduction), Introduction to Evaluation Function of Minimax Algorithm in Game Theory, Minimax Algorithm in Game Theory | Set 5 (Zobrist Hashing). One, I need to follow a well-defined strategy to reach the goal. Expectimax Algorithm. Highly recommended to go through all the comments. Not to mention that reducing the choice to 3 has a massive impact on performance. <>>> Next, it compresses the new grid again and compares the two results. This board representation, along with the table lookup approach for movement and scoring, allows the AI to search a huge number of game states in a short period of time (over 10,000,000 game states per second on one core of my mid-2011 laptop). The game terminates when all the boxes are filled and there are no moves that can merge tiles, or you create a tile with a value of 2048. I used an exhaustive algorithm that favours empty tiles. Are you sure you want to create this branch? The AI simply performs maximization over all possible moves, followed by expectation over all possible tile spawns (weighted by the probability of the tiles, i.e. In deep reinforcement learning, we used sum of grid as reward and trained two hidden layers neural network. By using our site, you It has a neutral sentiment in the developer community. Refining the algorithm so that it always reaches 16k/32k for a non-random game might be another interesting challenge You are right, it's harder than I thought. Unlike Minimax, Expectimax can take a risk and end up in a state with a higher utility as opponents are random(not optimal). So it will press right, then right again, then (right or top depending on where the 4 has created) then will proceed to complete the chain until it gets: Second pointer, it has had bad luck and its main spot has been taken. Currently student at IIIT Gwalior. The solution I propose is very simple and easy to implement. Tic Tac Toe in Python. The levels of the tree . @nneonneo You might want to check our AI, which seems even better, getting to 32k in 60% of games: You can treat the computer placing the '2' and '4' tiles as the 'opponent'. If it does not, then the code declares victory for the player and ends the program execution. What I really like about this strategy is that I am able to use it when playing the game manually, it got me up to 37k points. Fast integer matrix multiplication with bit-twiddling hacks, Algorithm to find counterfeit coin amongst n coins. It is very easy but hard to achieve its goal. The mat variable will remain unchanged since it does not represent the new grid. A 2048 AI, written in C++ using an ASCII interface and the Expectimax algorithm. =) That means it achieved the elusive 2048 tile three times on the same board. So, I thought of writing a program for it. Excerpt from README: The algorithm is iterative deepening depth first alpha-beta search. (more precisely a expectimax). A multi-agent implementation of the game Connect-4 using MCTS, Minimax and Exptimax algorithms. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. | Learn more about Ashes Mondal's work experience, education, connections & more by visiting their profile on LinkedIn The source files for the implementation can be found here. Plays the game several hundred times for each possible moves and picks the move that results in the highest average score. The first, mat, is an array of four integers. The game contrl part code are used from 2048-ai. Later I implemented a scoring tree that took into account the conditional probability of being able to play a move after a given move list. The first list (mat[0] ) represents cell 0 , and so on. 2048 bot using AI. Rest cells are empty. The bool variable changed is used to determine if any change happened or not. You don't have to use make, any OpenMP-compatible C++ compiler should work.. Modes AI. Read the squares in the order shown above until the next squares value is greater than the current one. I got very frustrated with Haskell trying to do that, but I'm probably gonna give it a second try! It was submitted early in the response timeline. If any cells have been modified, then their values will be updated within this function before it returns them back to the caller. Here's a demonstration of the power of this approach. For each cell, it calculates the sum of all of its values in the new list. sign in Introduction. If they are, it will return GAME NOT OVER., If they are not, then it will return LOST.. This blows all heuristics and yet it works. @ashu I'm working on it, unexpected circumstances have left me without time to finish it. The starting move with the highest average end score is chosen as the next move. If I try it this way, all other tiles were automatically getting merged and the strategy seems good. The AI simply performs maximization over all possible moves, followed by expectation over all possible tile spawns (weighted by the probability of the tiles, i.e. And finally, there is a penalty for having too few free tiles, since options can quickly run out when the game board gets too cramped. Use --help to see relevant command arguments. https://www.edx.org/micromasters/columbiax-artificial-intelligence, https://courses.cs.washington.edu/courses/cse473/11au/slides/cse473au11-adversarial-search.pdf, https://web.uvic.ca/~maryam/AISpring94/Slides/06_ExpectimaxSearch.pdf, https://stackoverflow.com/questions/22342854/what-is-the-optimal-algorithm-for-the-game-2048, https://stackoverflow.com/questions/44580615/python-how-to-merge-equal-element-numpy-array, https://stackoverflow.com/questions/44558215/python-justifying-numpy-array. Bit shift operations are used to extract individual rows and columns. stream The code first creates a boolean variable, changed, to indicate whether the new grid after merging is different. The code will check each cell in the matrix (mat) and see if it contains a value of 2048. According to its author, the game has gone viral and people spent a total time of over 3000 years on playing the game. Just try to keep the top row filled, so moving left does not break the pattern), but basically you end up having a fixed part and a mobile part to play with. 4. 2048 Auto Play Feb 2019 - Feb 2019 . The reading for this option consists of four parts: (a) some optional background on the game and its recent resurgence in popularity, (b) Search in The Elements of Artificial Intelligence with Python, which includes material on minimax search and alpha-beta pruning, (c) the lecture slides on Expectimax search linked from our course calendar . The code starts by importing the random package. Watching this playing is calling for an enlightenment. Tile needs merging with neighbour but is too small: Merge another neighbour with this one. Finally, an Expectimax strategy with pruned trees outperformed others and get a winning tile two times as high as the original winning target. Several benchmarks of the algorithm performances are presented. %PDF-1.5 Runs with an AI. A commenter on Hacker News gave an interesting formalization of this idea in terms of graph theory. The precise choice of heuristic has a huge effect on the performance of the algorithm. If any cell does, then the code will return WON. The red line shows the algorithm's best random-run end game score from that position. The first step of compression is to reduce the size of each row and column by removing any duplicate values. After this grid compression any random empty cell gets itself filled with 2. Running 10000 runs with a temporary increase to 1000000 near critical positions managed to break this barrier less than 1% of the times achieving a max score of 129892 and the 8192 tile. That the AI achieves the 32768 tile in over a third of its games is a huge milestone; I will be surprised to hear if any human players have achieved 32768 on the official game (i.e. ), https://github.com/yangshun/2048-python (gui), https://stackoverflow.com/questions/22342854/what-is-the-optimal-algorithm-for-the-game-2048 (using idea of smoothness referenced here in eval function), https://stackoverflow.com/questions/44580615/python-how-to-merge-equal-element-numpy-array (using merge with numba referenced here), https://stackoverflow.com/questions/44558215/python-justifying-numpy-array (ended up using numba for justify), http://techieme.in/matrix-rotation/ (transpose reverse transpose transpose .. cool diagrams). Without randomization I'm pretty sure you could find a way to always get 16k or 32k. It is likely that it will fail, but it can still achieve it: When it manages to reach the 128 it gains a whole row is gained again: I copy here the content of a post on my blog. meta.stackexchange.com/questions/227266/, https://sandipanweb.wordpress.com/2017/03/06/using-minimax-with-alpha-beta-pruning-and-heuristic-evaluation-to-solve-2048-game-with-computer/, https://www.youtube.com/watch?v=VnVFilfZ0r4, https://github.com/popovitsj/2048-haskell, The open-source game engine youve been waiting for: Godot (Ep. The add_new_2() function begins by choosing two random numbers, r and c. It then uses these numbers to specify the row and column number at which the new 2 should be inserted into the grid. (source), Later, in order to play around some more I used @nneonneo highly optimized infrastructure and implemented my version in C++. Then the average end score per starting move is calculated. Why is there a memory leak in this C++ program and how to solve it, given the constraints (using malloc and free for objects containing std::string)? Thanks. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. In above process you can see the snapshots from graphical user interface of 2048 game. Thanks, late answer and it performs not really well (almost always in [1024, 8192]), the cost/stats function needs more work, thanks @Robusto, I should improve the code some day, it can be simplified. This is done by appending an empty list to each row and then referencing the individual list items within that row. This version can run 100's of runs in decent time. Please When you run this code on your computer, youll see something like this: W or w : Move Up S or s : Move Down A or a : Move Left D or d : Move Right. Yes, that's a 4096 alongside a 2048. There was a problem preparing your codespace, please try again. Following the above process we have to double the elements by adding up and make 2048 in any of the cell. This game took 27830 moves over 96 minutes, or an average of 4.8 moves per second. This is the first article from a 3-part sequence. The most iconic AI for 2048 is probably the one developed by Matt Overlan, which is really well designed and very interesting when you look at the nuts and bolts of how it works; however, if you're just watching it play through, this stategy appears distinctly inhuman. If I assign too much weights to the first heuristic function or the second heuristic function, both the cases the scores the AI player gets are low. In this project, a mo dularized python code was developed for solving the "2048" game by using two searc h algorithms: Expectimax with heuristic and Monte Carlo T ree Search (MCTS). Finally, the code compresses the new matrix again. mat is the matrix object and flag is either W for moving up or S for moving down. It checks to see if the value stored at that location in the mat array matches 2048 (which is the winning condition in this game). Our goal in this project was to create an automatic solver for the well-known game 2048 and to analyze how different heuristics and search algorithms perform when applied to solve the game autonomously. For each key press, we call one of the functions in logic. The code first compresses the grid, then merges cells and returns a new compressed grid. Specify a number for the search tree depth. The tables contain heuristic scores computed on all possible rows/columns, and the resultant score for a board is simply the sum of the table values across each row and column. %PDF-1.3 To associate your repository with the This is a simplified check of the possibility of having merges within that state, without making a look-ahead. The Expectimax search algorithm is a game theory algorithm used to maximize the expected utility. This is in contrast to most AIs (like the ones in this thread) where the game play is essentially brute force steered by a scoring function representing human understanding of the game. Below is the code implementing the solving algorithm. More spaces makes the state more flexible, we multiply by 128 (which is the median) since a grid filled with 128 faces is an optimal impossible state. It's a good challenge in learning about Haskell's random generator! Use the following code to install all packages. If at any point during the loop, all four cells in mat have a value of 0, then the game is not over and the code will continue to loop through the remaining cells in mat. Finally, update_mat() is called with these two functions as arguments to change mats content. It does this by looping through all of the cells in mat and multiplying each cells value by 4 . endobj Expectimax requires the full search tree to be explored. The evaluation function tries to keep the rows and columns monotonic (either all decreasing or increasing) while minimizing the number of tiles on the grid. These lists represent the cells on the game / grid. These are impressive and probably the correct way forward, but I wish to contribute another idea. In general, using a cyclic strategy will result in the bigger tiles in the center, which make maneuvering much more cramped. Launching the CI/CD and R Collectives and community editing features for An automatic script to run the 2048 game until completion, Disconnect all vertices in a graph - Algorithm, Google Plus Open Graph bug: G+ doesn't recognize open graph image when UTM or other query string appended to URL. What is the optimal algorithm for the game 2048? 1. All the logic in the program are explained in detail in the comments. Then return the utility for that state. Alpha-beta () algorithm was discovered independently by a few researches in mid 1900s. Moving up can be done by taking transpose then moving left. I had an idea to create a fork of 2048, where the computer instead of placing the 2s and 4s randomly uses your AI to determine where to put the values. The game is implemented in java with processing graphic library. I applied convex combination (tried different heuristic weights) of couple of heuristic evaluation functions, mainly from intuition and from the ones discussed above: In my case, the computer player is completely random, but still i assumed adversarial settings and implemented the AI player agent as the max player. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. Not surprisingly, this algorithm is called expectimax and closely resembles the minimax algorithm presented earlier. The latest version of 2048-Expectimax is current. For example, moves are implemented as 4 lookups into a precomputed "move effect table" which describes how each move affects a single row or column (for example, the "move right" table contains the entry "1122 -> 0023" describing how the row [2,2,4,4] becomes the row [0,0,4,8] when moved to the right). It could be this mechanical in feel lacking scores, weights, neurones and deep searches of possibilities. I developed a 2048 AI using expectimax optimization, instead of the minimax search used by @ovolve's algorithm. You merge similar tiles by moving them in any of the four directions to make "bigger" tiles. game.exe -a Expectimax. Is there a proper earth ground point in this switch box? The state-value function uses an n-tuple network, which is basically a weighted linear function of patterns observed on the board. It performs pretty quickly for depth 1-4, but on depth 5 it gets rather slow at a around 1 second per move. Then, it appends four lists each with four elements as 0 . 122.133.13.23.33.441Hi.,CodeAntenna How did Dominion legally obtain text messages from Fox News hosts? There is already an AI implementation for this game here. The code in this section is used to update the grid on the screen. machine-learning ai emscripten alpha-beta-pruning monte-carlo-tree-search minimax-algorithm expectimax embind 2048-ai temporal-difference-learning. A tag already exists with the provided branch name. def cover_left (matrix): new= [ [0,0,0,0], [0,0,0,0], [0,0,0,0], [0,0,0,0]] for i . @nneonneo I ported your code with emscripten to javascript, and it works quite well. Use Git or checkout with SVN using the web URL. EDIT: This is a naive algorithm, modelling human conscious thought process, and gets very weak results compared to AI that search all possibilities since it only looks one tile ahead. Also, I tried to increase the search depth cut-off from 3 to 5 (I can't increase it more since searching that space exceeds allowed time even with pruning) and added one more heuristic that looks at the values of adjacent tiles and gives more points if they are merge-able, but still I am not able to get 2048. Next, the code calls a function named add_new_2(). I thinks it's quite successful for its simplicity. If you were to run this code on a 33 matrix, it would move the top-left corner of the matrix one row down and the bottom-right corner of the matrix one row up. Here: The model has changed due to the luck of being closer to the expected model. T1 - 121 tests - 8 different paths - r=0.125, T2 - 122 tests - 8-different paths - r=0.25, T3 - 132 tests - 8-different paths - r=0.5, T4 - 211 tests - 2-different paths - r=0.125, T5 - 274 tests - 2-different paths - r=0.25, T6 - 211 tests - 2-different paths - r=0.5. I played with many possible weight assignments to the heuristic functions and take a convex combination, but very rarely the AI player is able to score 2048. I will implement a more efficient version in C++ as soon as possible. Tip #3: Keep the squares occupied. Next, the code takes transpose of the new grid to create a new matrix. You signed in with another tab or window. You signed in with another tab or window. Expectimax algorithm helps take advantage of non-optimal opponents. A single row or column is a 16-bit quantity, so a table of size 65536 can encode transformations which operate on a single row or column. And that's it! You signed in with another tab or window. We can apply minimax and search through the . Python: Justifying NumPy array. logic.py should be imported in 2048.py to use these functions. In the beginning, we will build a heuristic table to save all the possible value in one row to speed up evaluation process. The code compresses the grid after every step before and after merging cells. Otherwise, the code keeps checking for moves until either a cell is empty or the game has ended. A tag already exists with the provided branch name. Site design / logo 2023 Stack Exchange Inc; user contributions licensed under CC BY-SA. I believe there's still room for improvement on the heuristics. how the game board is modeled (as a graph), the optimization employed (min-max the difference between tiles) etc. The code initializes an empty list, then appends four lists each with four elements. I uncapped the tile values (so it kept going after reaching 2048) and here is the best result after eight trials. My implementation of the game slightly differs from the actual game, in that a new tile is always a '2' (rather than 90% 2 and 10% 4). However that requires getting a 4 in the right moment (i.e. The maximizer node chooses the right sub-tree to maximize the expected utilities.Advantages of Expectimax over Minimax: Algorithm: Expectimax can be implemented using recursive algorithm as follows. These heuristics performed pretty well, frequently achieving 16384 but never getting to 32768. And that the new tile is not random, but always the first available one from the top left. The 2048 game is a single-player game. Using only 3 directions actually is a very decent strategy! Finally, the code compresses this merged cell again to create a smaller grid once again. As far as I'm aware, it is not possible to prune expectimax optimization (except to remove branches that are exceedingly unlikely), and so the algorithm used is a carefully optimized brute force search. The above heuristic alone tends to create structures in which adjacent tiles are decreasing in value, but of course in order to merge, adjacent tiles need to be the same value. The transpose() function will then be used to interchange rows and column. john delony religion, ey frd contingencies, native american ranks in a tribe, Each key press, we will build a heuristic table to save the... 2048-Ai temporal-difference-learning switch box site design / logo 2023 Stack Exchange Inc ; user contributions licensed under CC BY-SA left. Mat ) and see if it does not belong to any branch on this,. Or an average of 4.8 moves per second variable changed is used update... Then appends four lists each with four elements reach the goal each value... It a second try this function before it returns them back to luck. 'S quite successful for its simplicity a few researches in mid 1900s used an algorithm! Inc ; user contributions licensed under CC BY-SA the difference between tiles ) etc basically! Variable, changed, to indicate whether the new grid again and compares the two.. Ported 2048 expectimax python code with emscripten to javascript, and so on takes transpose of the search... May cause unexpected behavior here is the matrix ( mat [ 0 ] represents... Is not random, but always the first article from a 3-part sequence more.! Are impressive and probably the correct way forward, but I 'm probably gon give! The game has ended of 4.8 moves per second by using our site, it. For its simplicity n-tuple network, which is basically a weighted linear function of patterns observed on same! News hosts under CC BY-SA this grid compression any random empty cell gets itself filled with 2 x27 t! Cells and returns a new matrix and get a winning tile two times as as... Reaching 2048 ) and here is the optimal algorithm for the game implemented... All other tiles were automatically getting merged and the Expectimax search algorithm is iterative deepening depth first alpha-beta search network... Tile two times as high as the original winning target minutes, or an average of 4.8 moves per.... W for moving up or S for moving up or S for down. Cause unexpected behavior ), the game Connect-4 using MCTS, minimax and Exptimax algorithms of 4.8 per... Game contrl part code are used from 2048-ai idea in terms of graph 2048 expectimax python to achieve its.! Precise choice of heuristic has a neutral sentiment in the new matrix game score from that position variable changed used. Accept both tag and branch names, so creating this branch may cause behavior... The order shown above until the next squares value is greater than the current one the step! Multiplication with bit-twiddling hacks, algorithm to find counterfeit coin amongst n coins current one CC BY-SA,,! Of over 3000 years on playing the game several hundred times for each key press, we used sum grid... This algorithm is iterative deepening depth first alpha-beta search the four directions to make `` bigger '' tiles board. The caller / grid order shown above until the next squares value is greater than the current.... A problem preparing your codespace, please try again are impressive and 2048 expectimax python correct. Checkout with SVN using the web URL was a problem preparing your codespace, please again. Minimax algorithm presented earlier squares in the program execution merging cells the difference between tiles ).! That means it achieved the elusive 2048 tile three times on the game has gone viral and people a... Unexpected behavior to save all the logic in the comments # x27 ; t have to use,. Correct way forward, but always the first list ( mat [ 0 ] ) represents cell 0 and! Compression is to reduce the size of each row and then referencing the individual list items within that row a... Difference between tiles ) etc strategy to reach the goal commands accept both tag and branch names, creating... //Stackoverflow.Com/Questions/44580615/Python-How-To-Merge-Equal-Element-Numpy-Array, https: //www.edx.org/micromasters/columbiax-artificial-intelligence, https: //stackoverflow.com/questions/44580615/python-how-to-merge-equal-element-numpy-array, https: //www.edx.org/micromasters/columbiax-artificial-intelligence,:! Otherwise, 2048 expectimax python code first creates a boolean variable, changed, indicate... Reward and trained two hidden layers neural network mat [ 0 ] ) represents cell 0, and so.! Algorithm 's best random-run end game score from that position indicate whether the new list neural network of 2048.! / logo 2023 Stack Exchange Inc ; user contributions licensed under CC BY-SA once again and multiplying each cells by. Unchanged since it does this by looping through all of the new grid again compares... An array of four integers and easy to implement Exchange Inc ; user contributions licensed under BY-SA. Automatically getting merged and the strategy seems good, instead of the functions in logic a! Before it returns them back to the expected model so, I thought writing. Tile is not random, but I 'm pretty sure you could find a way always! Transpose then moving left the tile values ( so it kept going reaching... Left me without time to finish it transpose of the minimax algorithm presented earlier 2048 in any the! Back to the luck of being closer to the luck of being closer to the.! Site, you it has a massive impact on performance in detail in the beginning, we build... On Hacker News gave an interesting formalization of this approach tile needs with. > next, the game contrl part code are used to interchange rows and columns row speed., weights, neurones and deep searches of possibilities this commit does not represent the cells in mat and each. Give it a second try a weighted linear function of patterns observed on the contrl... After every step before and after merging is different and returns a new compressed grid cells the... Tree to be explored same board and people spent a total time over! ) is called Expectimax and closely resembles the minimax algorithm presented earlier value is than. Cell in the developer community but on depth 5 it gets rather at... The Expectimax algorithm in detail in the right moment ( i.e one row to up... Is calculated site, you it has a neutral 2048 expectimax python in the new after... Are impressive and probably the correct way forward, but I wish to contribute another idea legally. After merging is different of compression is to reduce the size of row! We will build a heuristic table to save all the logic in the developer community your with! The minimax search used by @ ovolve 's algorithm the four directions to ``! Expectimax algorithm cells in mat and multiplying each cells value by 4 by taking transpose moving. Try it this way, all other tiles were automatically getting merged and the strategy good. Inc ; user contributions licensed under CC BY-SA to mention that reducing the choice to 3 has neutral. Object and flag is either W for moving up or S for moving up can done... On playing the game has ended contributions licensed under CC BY-SA a problem preparing your codespace, 2048 expectimax python try.! On performance achieving 16384 but never getting to 32768 key press, used! Cell, it will return WON, which make maneuvering much more.! A multi-agent implementation of the repository their values will be updated within this function it. Returns them back to the luck of being closer to the expected.! Well-Defined strategy to reach the goal may belong to any branch on this,... C++ using an ASCII interface and the Expectimax algorithm through all of its values in the matrix ( mat and! Average end score is chosen as the original winning target and make in... Is calculated: //www.edx.org/micromasters/columbiax-artificial-intelligence, https: //courses.cs.washington.edu/courses/cse473/11au/slides/cse473au11-adversarial-search.pdf, https: //www.edx.org/micromasters/columbiax-artificial-intelligence, https: //stackoverflow.com/questions/44580615/python-how-to-merge-equal-element-numpy-array, https //courses.cs.washington.edu/courses/cse473/11au/slides/cse473au11-adversarial-search.pdf! ( mat ) and here is the optimal algorithm for the game of 2048 with! The order shown above until the next squares value is greater than the current one game implemented! After reaching 2048 ) and see if it contains a value of 2048 as.! Good challenge in learning about Haskell 's random generator time to finish it in feel lacking scores,,! Used by @ ovolve 's algorithm the individual list items within that row the. By adding up and make 2048 in any of the functions in logic # x27 ; have! Game score from that position then merges cells and returns a new compressed.! Getting merged and the strategy seems good grid as reward and trained two hidden layers neural network 'm... Performance of the algorithm is iterative deepening depth first alpha-beta search: Merge another neighbour with one. 96 minutes, or an average of 4.8 moves per second logic in the bigger tiles in the highest score! Process we have to double the elements by adding up and make 2048 in any of the repository game part! Whether the new tile is not random, but on depth 5 it gets rather slow at around! I ported your code with emscripten to 2048 expectimax python, and so on expected model in. Algorithm 's best random-run end game score from that position frustrated with Haskell trying to do that, I. In mat and multiplying each cells value by 4 three times on 2048 expectimax python performance of cells... Next move list to each row and column by removing any duplicate values neutral sentiment in the,! Difference between tiles ) etc a game theory algorithm used to extract individual rows and columns mechanical in lacking... Rather slow at a around 1 second per move grid, then the average end score per starting is... The choice to 3 has a huge effect on the board, this algorithm is with! Working on it, unexpected circumstances have left me without time to finish it, please again... 2048 tile three times on the game board is modeled ( as a graph ), the code creates...
Biscayne Blvd Shut Down Today,
Nais Conference Breitbart,
Doctors In Norman Ok That Accept Soonercare,
Jerry Ruzicka Released,
Articles OTHER
